AIoT Hub - (Re)-engineering AI

The widespread deployment of AI beyond the cloud and onto smaller, resource-constrained devices presents a fundamental engineering challenge. While considerable attention has focused on developing increasingly capable models, making ML systems practical in real-world environments requires us to rethink how we engineer them.
In this talk, I will discuss three fundamental challenges that arise when deploying ML systems at scale: they do not run efficiently on small devices, they have inherent sequential dependencies that limit efficient execution, and they are highly memory-intensive. I will present research solutions we have developed to address these challenges, drawing on our experience in designing and deploying distributed and edge AI systems.
Rather than focusing on a particular model or application, the talk will explore the underlying systems problems and the engineering principles behind our solutions. I will show how rethinking the way ML systems are designed and executed can make AI more practical for deployment across resource-constrained environments.