On-Device & Edge Hardware

Not all AI needs to run in a data centre. On-device AI - running models directly on phones, laptops, cars, cameras, or IoT sensors - offers advantages in latency, privacy, and offline capability. Your phone's face unlock, voice assistant wake words, and camera scene detection all use on-device AI. The hardware enabling this includes dedicated neural processing units (NPUs) built into mobile chips from Apple, Qualcomm, and MediaTek; AI-capable microcontrollers for IoT devices; and increasingly powerful laptop and desktop processors with integrated AI accelerators. Apple's Neural Engine, Qualcomm's Hexagon NPU, and Intel's and AMD's latest laptop chips all include hardware specifically for running AI workloads locally. The challenge with edge AI is working within tight constraints - limited power, memory, and processing capability compared to the data centre. Models need to be small and efficient, which is why techniques like quantisation, pruning, and knowledge distillation are so important for edge deployment. The trend is clearly toward more AI running locally. Microsoft's Copilot+ PCs, Apple Intelligence features, and Google's on-device language models all reflect this direction. For applications where privacy is paramount, latency is critical, or connectivity is unreliable, on-device AI is not just convenient - it's necessary.