Edge AI System

An Edge AI System is a distributed computing architecture that deploys trained machine learning models directly onto edge devices, sensors, and gateways at the network periphery, enabling local inference without requiring continuous cloud connectivity. Processing occurs close to the data source, achieving sub-millisecond latency, reduced bandwidth consumption, and enhanced data privacy. Edge AI Systems must fit within the memory, power, and computational constraints of embedded hardware through model compression techniques such as quantisation, pruning, and knowledge distillation.

Semantic Classification

Content

Compositional Relationships (Components)

SubClassOf(ai:EdgeAISystem
  ObjectSomeValuesFrom(ai:hasPart ai:EdgeAIAccelerator))
SubClassOf(ai:EdgeAISystem
  ObjectSomeValuesFrom(ai:hasPart ai:EmbeddedSystems))
SubClassOf(ai:EdgeAISystem
  ObjectSomeValuesFrom(ai:hasPart ai:Sensor))
SubClassOf(ai:EdgeAISystem
  ObjectSomeValuesFrom(ai:hasPart ai:NeuralProcessingUnit))
SubClassOf(ai:EdgeAISystem
  ObjectSomeValuesFrom(ai:hasPart ai:IoTGateway))
SubClassOf(ai:EdgeAISystem
  ObjectSomeValuesFrom(ai:hasPart ai:ModelCompressionForEdge))
SubClassOf(ai:EdgeAISystem
  ObjectSomeValuesFrom(ai:hasPart ai:OnDeviceLearning))

Dependency Relationships

SubClassOf(ai:EdgeAISystem
  ObjectSomeValuesFrom(ai:requires ai:NeuralNetworkQuantisation))
SubClassOf(ai:EdgeAISystem
  ObjectSomeValuesFrom(ai:requires ai:ModelCompressionForEdge))
SubClassOf(ai:EdgeAISystem
  ObjectSomeValuesFrom(ai:requires ai:KnowledgeDistillation))
SubClassOf(ai:EdgeAISystem
  ObjectSomeValuesFrom(ai:requires ai:RealTimeInferenceAtEdge))
SubClassOf(ai:EdgeAISystem
  ObjectSomeValuesFrom(ai:dependsOn ai:EdgeComputing))
SubClassOf(ai:EdgeAISystem
  ObjectSomeValuesFrom(ai:dependsOn ai:MachineLearning))
SubClassOf(ai:EdgeAISystem
  ObjectSomeValuesFrom(ai:dependsOn ai:EmbeddedSystems))

Capability Relationships

SubClassOf(ai:EdgeAISystem
  ObjectSomeValuesFrom(ai:enables ai:AutonomousRobot))
SubClassOf(ai:EdgeAISystem
  ObjectSomeValuesFrom(ai:enables ai:PredictiveMaintenance))
SubClassOf(ai:EdgeAISystem
  ObjectSomeValuesFrom(ai:enables ai:SmartCity))
SubClassOf(ai:EdgeAISystem
  ObjectSomeValuesFrom(ai:enables ai:IndustrialAutomation))
SubClassOf(ai:EdgeAISystem
  ObjectSomeValuesFrom(ai:enables ai:OnDeviceLearning))
SubClassOf(ai:EdgeAISystem
  ObjectSomeValuesFrom(ai:enables ai:FederatedLearning))
SubClassOf(ai:EdgeAISystem
  ObjectSomeValuesFrom(ai:enables ai:ComputerVision))

Implementation Relationships

SubClassOf(ai:EdgeAISystem
  ObjectSomeValuesFrom(ai:implements ai:FederatedLearning))
SubClassOf(ai:EdgeAISystem
  ObjectSomeValuesFrom(ai:implements ai:TinyML))
SubClassOf(ai:EdgeAISystem
  ObjectSomeValuesFrom(ai:implements ai:ModelCompressionForEdge))
SubClassOf(ai:EdgeAISystem
  ObjectSomeValuesFrom(ai:uses ai:EmbeddedAIFrameworks))
SubClassOf(ai:EdgeAISystem
  ObjectSomeValuesFrom(ai:uses ai:NeuralNetworkQuantisation))
SubClassOf(ai:EdgeAISystem
  ObjectSomeValuesFrom(ai:uses ai:KnowledgeDistillation))
SubClassOf(ai:EdgeAISystem
  ObjectSomeValuesFrom(ai:uses ai:TensorFlowLite))
SubClassOf(ai:EdgeAISystem
  ObjectSomeValuesFrom(ai:uses ai:ONNXRuntime))

Reduction Relationships

SubClassOf(ai:EdgeAISystem
  ObjectSomeValuesFrom(ai:reducesTo ai:EmbeddedSystems))
SubClassOf(ai:EdgeAISystem
  ObjectSomeValuesFrom(ai:reducesTo ai:TinyMLDeployment))
SubClassOf(ai:EdgeAISystem
  ObjectSomeValuesFrom(ai:reducesTo ai:IoTGateway))

Support and Contrast Axioms

SubClassOf(ai:EdgeAISystem
  ObjectSomeValuesFrom(ai:supports ai:InternetOfThings))
SubClassOf(ai:EdgeAISystem
  ObjectSomeValuesFrom(ai:supports ai:DigitalTwin))
SubClassOf(ai:EdgeAISystem
  ObjectSomeValuesFrom(ai:supports ai:IndustrialAutomation))
SubClassOf(ai:EdgeAISystem
  ObjectSomeValuesFrom(ai:bridgesTo ai:FederatedLearning))
SubClassOf(ai:EdgeAISystem
  ObjectSomeValuesFrom(ai:bridgesTo ai:CyberPhysicalSystems))
SubClassOf(ai:EdgeAISystem
  ObjectSomeValuesFrom(ai:standardizedBy ai:IEEE))
SubClassOf(ai:EdgeAISystem
  ObjectSomeValuesFrom(ai:standardizedBy ai:ARMHoldings))

About

• Edge AI Systems represent the confluence of three simultaneously maturing technological trajectories: the proliferation of heterogeneous edge hardware with dedicated neural processing capabilities, advances in model compression that shrink state-of-the-art deep networks from billions of parameters to kilobyte-scale deployments, and growing regulatory, commercial, and safety pressure to process sensitive data locally rather than transmitting it to remote server infrastructure. The result is an architectural paradigm in which trained Machine Learning models migrate from the data centre to the device — where data originates, where actuation must occur, and where latency, bandwidth, and privacy constraints are most acute. This architectural shift is both technically demanding (operating at the hardware-software interface of Embedded Systems, RTOS, and compressed neural network execution) and commercially significant: the global Edge AI Hardware market was valued at USD 25.08 billion in 2025, growing at approximately 18% CAGR toward a projected USD 68.73 billion by 2031, with the industrial IoT sector showing the fastest growth driven by Industry 4.0 automation.
• The foundational motivation for Edge AI is the recognition that the cloud-centric inference model — in which every decision requires a round-trip to a remote inference server — is structurally incompatible with many real-world deployment requirements. Autonomous vehicles cannot tolerate the 50-100ms round-trip latency of cloud inference for safety-critical perception decisions; industrial equipment vibration sensors must detect anomalous signatures in microseconds to engage protective interlocks before damage occurs; healthcare wearables monitoring cardiac arrhythmias must process ECG signals continuously without transmitting personal biometric streams; smart cameras performing access control must authenticate without sending facial imagery to external servers. In each case, the combination of latency constraints, network unreliability, data privacy obligations, and bandwidth costs makes on-device or near-device inference the only viable operational model. Edge AI Systems formalise the architecture, compression workflow, and hardware–software integration required to make this local inference possible across the full range of device classes, from sub-milliwatt microcontrollers to embedded GPU platforms.
• The technical core of Edge AI deployment is model compression: the systematic reduction of a trained neural network’s resource footprint to fit within edge hardware constraints while preserving acceptable inference accuracy. This is not a single technique but a pipeline of complementary methods applied sequentially or jointly. Neural Network Quantisation reduces numerical precision of weights and activations from 32-bit floating point (FP32) to 8-bit integers (INT8), 4-bit integers (INT4), or in extreme cases binary (1-bit) representations, yielding 4x-32x reductions in memory footprint and proportional inference speedup on hardware with integer arithmetic units. Post-training quantisation applies this without retraining; quantisation-aware training incorporates the quantisation error into the training loss, recovering accuracy lost at aggressive precision targets. Structured pruning removes entire convolutional filters, attention heads, or transformer blocks whose outputs are near-zero under the deployment workload, producing architecturally simpler networks that run efficiently on standard hardware without requiring special sparse matrix kernels. Knowledge Distillation trains a compact student network to reproduce the soft output distribution (logit vectors) of a larger teacher network rather than matching hard ground-truth labels, leveraging the teacher’s rich intermediate representations to guide efficient generalisation in a model orders of magnitude smaller. Hardware-aware Neural Architecture Search (NAS) automates the co-design of network topology and target hardware constraints, producing architectures that are Pareto-optimal in the accuracy-latency-memory trade-off space for a specific target device, as pioneered by MobileNet (Howard et al., Google, 2017), EfficientNet (Tan and Le, Google, 2019), and MobileNetV4 (2024).
• Federated Edge Learning extends Edge AI from pure inference to include on-device model adaptation, addressing the fundamental limitation that a model trained on centralised data may perform poorly on the heterogeneous local distributions encountered on deployed edge devices. Under federated learning (McMahan et al., Google, 2017), each edge device trains locally on its private data and computes a gradient update, which is transmitted — encrypted and optionally differentially private — to a secure aggregation server. The server aggregates updates across the fleet (using FedAvg, FedProx, or personalised federation variants), broadcasts the improved global model, and the cycle repeats. This pattern is critical in healthcare wearables (where each patient’s physiological patterns differ), industrial predictive maintenance (where each machine’s operating profile is unique), and personalised on-device language models (where each user’s communication style and vocabulary differ). Edge-Cloud Collaboration architectures complement purely local operation by routing high-confidence, low-complexity decisions to the edge while escalating ambiguous inputs to richer cloud models — implementing an intelligent triage that balances latency, accuracy, and compute cost dynamically at runtime.

Components / Architecture

• Hardware substrate: The diversity of Edge AI hardware reflects the equally diverse range of deployment requirements, from nano-watt keyword-spotting chips embedded in hearing aids to 100-watt embedded AI platforms in autonomous vehicles. Ultra-low-power microcontrollers (ARM Cortex-M0 through M55, RISC-V, ESP32) operating at 10-500 MHz with kilobytes to low megabytes of SRAM and flash support TinyML workloads — keyword spotting, simple gesture recognition, anomaly detection — using quantised networks with tens of thousands of parameters. Cortex-A class application processors (ARM Cortex-A55 through A720) with integrated NPUs on mobile SoCs (Qualcomm Snapdragon X series, MediaTek Dimensity, Apple A-series, Samsung Exynos) support mobile-tier inference for Computer Vision, Natural Language Processing, and multimodal perception at 0.7-1.2 mJ/inference — 3-8x more energy-efficient than mobile CPU execution for the same workload. NVIDIA Jetson-class embedded GPU modules (Orin, AGX, Xavier lineages) provide 10-275 TOPS of INT8 throughput for robotics, autonomous vehicles, and industrial vision systems requiring large-model inference at camera-frame rates. Dedicated edge AI ASICs (Google Coral Edge TPU, Hailo-8/15 NPU, Kneron, Mythic Analog Compute-in-Memory) optimise for specific workloads with power envelopes of 1-8 watts, delivering TOPS/W figures exceeding GPU alternatives by large margins. The choice of hardware determines permissible model size, quantisation depth, and framework support, making hardware-model co-design essential to Edge AI system architecture.
• Model compression pipeline: The engineering workflow that transforms a cloud-scale trained model into an edge-deployable artefact. Starting from a trained floating-point model (FP32 or FP16), the pipeline applies: (1) architecture selection or NAS — choosing a backbone (MobileNetV4, EfficientNet-Lite, YOLOv8n, Whisper Tiny, Phi-3-mini) matched to the target device’s compute budget; (2) Knowledge Distillation if the target architecture is substantially smaller than the best available teacher; (3) structured pruning — iterative magnitude-based filter removal with fine-tuning cycles to recover accuracy loss; (4) Neural Network Quantisation — post-training quantisation (PTQ) for fast compression, or quantisation-aware training (QAT) for higher accuracy at aggressive bit-widths; (5) operator fusion and memory layout optimisation applied by the deployment runtime (TensorFlow Lite converter, ONNX Runtime, OpenVINO). The output is a compiled, quantised model artefact (.tflite, .onnx, .blob, .axmodel) with known memory footprint, inference latency on the target hardware, and validated accuracy metrics from a held-out test set representative of the deployment distribution. Hybrid approaches combining pruning, quantisation, and distillation in a single fine-tuning phase (2025) achieve compression ratios of 10-100x with accuracy degradation under 2% on standard benchmarks.
• Embedded AI frameworks: The software runtime layer that abstracts over heterogeneous edge hardware, providing model loading, operator scheduling, memory management, and hardware accelerator dispatch. TensorFlow Lite (Google) — the most widely deployed edge AI runtime globally — provides a portable FlatBuffer model format, operator kernels for CPU (with ARM NEON SIMD acceleration) and GPU delegates, and hardware-specific delegates for Edge TPU, Hexagon DSP, and NPU backends; supports FP32, FP16, INT8, and dynamic-range quantisation. ONNX Runtime (Microsoft) — cross-platform runtime for the Open Neural Network Exchange format — supports a broad matrix of hardware via Execution Providers (CUDA, TensorRT, OpenVINO, CoreML, XNNPACK, QNN for Qualcomm), enabling write-once deploy-anywhere model portability. OpenVINO (Intel) — optimisation and inference toolkit targeting Intel silicon (CPU, integrated GPU, VPU, Movidius Neural Compute Stick) with FP16 and INT8 quantisation; widely used in industrial vision applications. TensorFlow Lite Micro (TFLM) — stripped-down runtime for MCUs with no OS, no dynamic memory allocation, and kilobyte RAM budgets; supports ARM Cortex-M, RISC-V, and Xtensa architectures. ExecuTorch (Meta/PyTorch) — emerging modular export and runtime framework targeting heterogeneous edge backends from the PyTorch model authoring ecosystem. Framework selection is determined by the hardware vendor ecosystem, required operator coverage, and the compression tools available in each framework’s conversion pipeline.
• On-device learning and federated adaptation: Static deployed models degrade in accuracy as real-world data distributions shift from the training distribution (covariate shift, concept drift). On-device learning addresses this through: (1) transfer learning fine-tuning — adapting a subset of model parameters (typically the final classification layer or lightweight adapter modules) on device-local data without full retraining; (2) federated learning — coordinated cross-device gradient aggregation for fleet-wide model improvement without centralising raw data; (3) continual learning — updating model parameters sequentially on new data while mitigating catastrophic forgetting of previous knowledge (elastic weight consolidation, progressive networks, replay buffers). On-device learning imposes additional compute and memory requirements beyond inference, requiring hardware with sufficient SRAM for gradient tensors and a fine-tuning optimisation loop — feasible on Cortex-A class and above, challenging on Cortex-M class without specialised approximate backpropagation methods.
• Data privacy and security architecture: Edge AI Systems are architectural responses to data sovereignty constraints as much as to latency constraints. By keeping raw sensor data on the device, they avoid the privacy risks of transmitting personal biometric, behavioural, or environmental data to external servers — directly addressing requirements under GDPR (EU), UK Data Protection Act 2018, and sector-specific regulations (NHS DSP Toolkit for health data, HIPAA for US healthcare). Secure execution environments (TrustZone on ARM, TEEs — Trusted Execution Environments) protect model weights and inference pipelines from malicious software on the host device, preventing model extraction attacks. Differential privacy (DP) mechanisms — adding calibrated Gaussian or Laplacian noise to gradient updates before federation — provide formal (ε, δ)-DP guarantees on the information leaked about individual training samples through the federated update process. Secure aggregation protocols (Bonawitz et al., Google, 2017) use cryptographic masking to prevent the aggregation server from observing individual device updates, providing additional privacy beyond DP alone. Model encryption (AES-128 or AES-256 of model weights at rest) prevents intellectual property extraction from physically accessible edge devices.
• Edge-Cloud Collaboration and tiered architectures: Pure on-device inference and full cloud inference are extremes of a continuum, with practical Edge AI Systems often implementing tiered inference architectures that dynamically allocate computation across the edge-fog-cloud hierarchy. Cascaded classifiers route easy cases (high confidence, low complexity) to a small fast on-device model, escalating ambiguous cases to a more capable but more expensive model — locally on a gateway, or in the cloud. Early-exit neural networks (BranchyNet, Shallow-Deep Networks) implement this cascade within a single model by adding exit branches at intermediate layers that output a prediction when confidence exceeds a threshold, avoiding the computational cost of the later layers for easy inputs. Split computing partitions a model at a layer boundary, executing early layers on device and later layers on a gateway or cloud server, transmitting only the compressed intermediate feature map rather than the raw input — reducing bandwidth by 10-100x versus raw data transmission while enabling access to larger models than the device can accommodate alone. Intelligent offloading controllers make real-time decisions about where to execute each inference based on current network latency, server load, device battery state, and privacy classification of the input data.

Use Cases / Major Families

• Industrial IoT and predictive maintenance: Edge AI Systems embedded in industrial equipment monitor vibration signatures, temperature gradients, current waveforms, and acoustic emissions from rotating machinery in real time, running anomaly detection models locally to predict bearing failure, motor imbalance, or lubrication degradation days before catastrophic failure occurs. The Siemens MindSphere ecosystem, Bosch ctrlX IoT, and ABB Ability platforms deploy edge AI inference at the factory floor level, with models retrained on equipment-specific operating data to achieve detection sensitivity impossible with generic cloud models. The UK’s Advanced Manufacturing Research Centre (AMRC) in Sheffield operates edge AI predictive maintenance pilots across aerospace and automotive supply chain partners. Vibration-anomaly detection at 400 kHz sample rates requires on-device FFT and CNN inference with sub-millisecond latency incompatible with cloud round-trips. Edge AI here reduces unplanned downtime (estimated at USD 50 billion annually across the aerospace, energy, and automotive sectors), lowers maintenance labour costs, and enables continuous monitoring of assets in environments with unreliable connectivity.
• Healthcare wearables and remote patient monitoring: Wearable biosensors (smartwatches, ECG patches, continuous glucose monitors, pulse oximeters, neural implants) deploy compressed Machine Learning models to perform arrhythmia detection, seizure detection, hypoglycaemia prediction, and sleep staging locally on the wrist or body, without transmitting raw physiological streams to cloud servers. Apple’s atrial fibrillation detection (implemented in ECG app, Apple Watch Series 4+) runs a CNN on the watch’s S-series SiP using CoreML — a canonical Edge AI healthcare deployment reaching hundreds of millions of users. The NHS AI deployment programmes include remote monitoring of cardiac patients discharged after acute episodes, using certified edge AI devices to detect deterioration signals and generate alerts without requiring patient interaction. On-device sleep staging models (running on ARM Cortex-M class) consume under 1 mW in inference mode, enabling week-long continuous monitoring on coin cell batteries. The sensitivity of health data under GDPR and NHS DSP Toolkit makes the privacy-preserving properties of Edge AI inference architecturally mandatory, not merely beneficial.
• Autonomous vehicles and robotics: Self-driving perception stacks are the most computationally intensive Edge AI deployment class, running LiDAR point cloud segmentation, camera-based object detection (YOLO, DETR), radar-camera fusion, lane estimation, and occupancy grid prediction simultaneously at 10-30 fps with end-to-end latency under 50ms. NVIDIA Drive AGX platforms (Orin, 254 TOPS), Qualcomm Snapdragon Ride, and Mobileye EyeQ series are the dominant hardware substrates, consuming 30-100W per compute unit. Autonomous Robot systems in warehouses (Boston Dynamics Spot, Amazon Kiva-class AMRs) and factories run on-board perception and navigation models using NVIDIA Jetson Orin or Xavier modules. The UK’s CAM Pathfinder programme (£150m, running to 2030) funds connected and autonomous vehicle development across university and industrial partners, with ARM Holdings contributing IP for the neural processing silicon used in automotive-grade SoCs. NCAP and ISO 26262 automotive safety standards require deterministic, verifiable inference performance that favours dedicated NPU hardware over general-purpose GPU execution with variable latency.
• Smart cameras and computer vision at the edge: Embedded Computer Vision is the highest-volume Edge AI application class, covering smart retail (shelf analytics, checkout-free stores, people counting), smart cities (traffic management, parking optimisation, environmental monitoring), security and access control (face liveness detection, behaviour anomaly detection), and quality control in manufacturing (surface defect detection on production lines at camera-frame rates). The Hailo-8 NPU achieves 26 TOPS at 2.8W for object detection and semantic segmentation tasks, enabling HD-resolution inference on cameras without external compute. OpenCV AI Kit (OAK-D), NVIDIA Jetson Nano, and Google Coral Developer Board exemplify accessible embedded vision platforms. Keyword spotting and wake-word detection on microcontrollers (ARM Cortex-M) represent the minimal edge AI deployment: running sub-100kB CNN or attention models continuously at under 1mW, activating a larger system only when a trigger word is detected.
• TinyML and ultra-low-power microcontroller AI: TinyML is the subdiscipline focused on deploying Machine Learning models on microcontrollers with RAM budgets of 64-512 KB and flash budgets of 256KB-4MB, operating at power envelopes of 1μW-100mW. Applications include: anomaly detection in industrial sensors (vibration, current), predictive maintenance on simple actuators, gesture recognition from IMU data, keyword spotting for voice UI, and environmental sensing (air quality classification from gas sensor arrays). TensorFlow Lite for Microcontrollers (TFLM) and the Arduino Nano 33 BLE Sense represent the accessibility frontier — enabling community-developed edge AI models without specialised hardware knowledge. The MCUNet (MIT, Lin et al., 2020) and MobileNetV4 families provide architectures specifically designed for MCU RAM constraints, achieving ImageNet-class accuracy in sub-512KB model footprints through NAS-optimised operator selection and memory-efficient activation reuse patterns.
• Natural language processing at the edge: The deployment of small language models and voice AI on edge devices is the most rapidly evolving frontier in 2025-2026. Whisper Tiny (39M parameters, 75MB FP32, approximately 40MB INT8) runs on mid-range smartphones for offline speech recognition. Phi-3-mini (3.8B parameters, quantised to 2.4GB INT4) achieves GPT-3.5-equivalent performance on reasoning benchmarks while running on mobile NPUs at 10-20 tokens/second — enabling private offline LLM inference on consumer devices. On-device keyboard prediction and autocorrect have used miniature language models for years (Swift keyboard on iOS uses transformer-based models in under 20MB). The demand for private, low-latency LLM inference without cloud exposure is driving rapid adoption of edge NLP, with Qualcomm AI Hub and NVIDIA Jetson providing optimised deployment paths for popular small language models (SLMs: Phi-3, Gemma 2B, Llama 3.2 1B/3B, Mistral 7B quantised to INT4).

Academic Context

• The theoretical foundations of Edge AI Systems span several disciplines that have converged over the past decade. Neural network compression theory — providing information-theoretic bounds on the trade-off between model size and accuracy, Rademacher complexity bounds on generalisation of compressed networks, and lottery ticket hypothesis (Frankle and Carlin, 2019, MIT) explaining why small subnetworks of large trained models can match full-network performance — provides the rigorous basis for pruning and distillation practice. Information bottleneck theory (Tishby and Zaslavsky, 2015) offers a theoretical framework for understanding what information is preserved through model compression. Knowledge Distillation was introduced by Hinton, Vinyals, and Dean (2015, Google Brain) as a method for model compression via soft-label training; the subsequent literature has produced relational knowledge distillation, feature-level distillation, self-distillation, and online distillation variants. Federated learning was introduced by McMahan et al. (Google, 2017) under the FedAvg algorithm; since 2020, the FL literature has expanded explosively with personalised FL (per-device fine-tuning while retaining global model benefits), differential privacy integration (Abadi et al., Google Brain, 2016 — DP-SGD), and communication-efficient compression methods for federated gradient updates.
• Hardware-aware NAS for edge deployment was established by Google’s MobileNet family (Howard et al., 2017, MobileNetV1; Sandler et al., 2018, MobileNetV2; Howard et al., 2019, MobileNetV3) and EfficientNet (Tan and Le, 2019), which introduced compound scaling of depth, width, and resolution as a principled method for NAS targeting accuracy-efficiency trade-offs. NetAdapt (Yang et al., MIT, 2018) and MCUNet (Lin et al., MIT, 2020) extended NAS to strict hardware constraints (specific MCU RAM/flash budgets). TinyML as a subfield was catalysed by Pete Warden (Google) and Daniel Situnayake’s textbook “TinyML: Machine Learning with TensorFlow Lite on Arduino and Ultra-Low-Power Microcontrollers” (2019), which established the field’s vocabulary and developer community. The MLPerf Tiny benchmark suite (Banbury et al., 2021) standardised performance comparison across edge and MCU hardware, enabling systematic evaluation of the accuracy-latency-energy trade-off across the device spectrum. Quantisation theory and practice were systematised by Nagel et al.’s survey (Qualcomm AI Research, 2021) and the comprehensive “A Survey of Quantization Methods for Efficient Neural Network Inference” (Gholami et al., UC Berkeley, 2021), providing the reference treatment of post-training quantisation, QAT, mixed-precision quantisation, and activation quantisation.

Current Landscape (2026)

• The Edge AI hardware market in 2026 is dominated by two complementary segments: mobile SoC NPUs (Qualcomm Snapdragon X series, Apple A18 Pro, MediaTek Dimensity 9400) bringing 40-80 TOPS of on-device AI to smartphones and tablets with power envelopes of 2-8W; and embedded edge AI modules (NVIDIA Jetson Orin AGX at 275 TOPS/60W, Hailo-15 at 400 TOPS/10W, Google Coral with Edge TPU) serving industrial, robotics, and infrastructure applications. Qualcomm and NVIDIA combined held over 20.8% market share in the Edge AI processor segment in 2024. ARM Holdings announced at Embedded World 2026 new IP for intelligent edge AI systems at scale, extending the Cortex-A and Ethos NPU families for automotive, infrastructure, and IoT applications. NVIDIA opened a world-class AI laboratory at ARM’s Cambridge headquarters in 2024-2025, co-investing in edge AI research covering healthcare, autonomous vehicles, robotics, and data science — a significant UK concentration of edge AI hardware IP development.
• The software ecosystem has reached deployment maturity. TensorFlow Lite and ONNX Runtime are established universal runtimes; Meta’s ExecuTorch (2024) is emerging as the PyTorch community’s preferred path for mobile and edge export. Qualcomm AI Hub (2024) provides cloud-based model optimisation and device-in-the-cloud testing for Snapdragon-deployed models. The hybrid compression workflow — combining quantisation, structured pruning, and distillation in a single toolchain pass — has become the industry standard, with dedicated optimisation services from Qualcomm (AI Model Efficiency Toolkit), NVIDIA (TensorRT), Intel (OpenVINO Model Optimizer), and ARM (NN Vela compiler for Ethos NPU). NPU benchmark data from 2024-2025 shows NPUs outperforming GPUs by 3.2x for LLM inference latency on mobile SoCs, while achieving 0.7-1.2 mJ/inference versus 3.5-6.0 mJ on mobile CPUs — confirming NPUs as the energy-efficiency-optimal inference substrate for sustained edge AI deployment. The ACM Computing Surveys paper “Empowering Edge Intelligence: A Comprehensive Survey on On-Device AI Models” (2025) provides the field’s current reference treatment, covering model compression, hardware, frameworks, and applications across the full edge device spectrum.
• Regulation is shaping Edge AI architecture decisions in the UK and EU. The EU AI Act (effective 2024-2026 phased implementation) classifies several categories of edge AI systems — notably biometric identification, emotion recognition in workplaces, and safety-critical autonomous systems — as high-risk or prohibited, requiring conformity assessments, technical documentation of compression methods and accuracy degradation, and post-market monitoring. The UK’s PSTI (Product Security and Telecommunications Infrastructure) Act 2022 mandates baseline security requirements for consumer IoT devices, including edge AI-enabled smart cameras and speakers, with enforcement from April 2024. The National Edge AI Hub (UK, founded early 2024) coordinates industry and academic edge AI capability, with vertical focus areas including defence, energy, aerospace, autonomous driving, healthcare, and 6G connectivity. ETSI EN 303 645 security requirements and ISO/IEC 30141 IoT reference architecture apply to Edge AI System deployments within the broader IoT ecosystem.

UK Context

• The UK occupies a structurally important position in global Edge AI through ARM Holdings, headquartered in Cambridge, whose processor IP underlies the vast majority of edge devices worldwide — every smartphone, virtually every microcontroller, and a large fraction of automotive and industrial SoCs run on ARM Cortex architecture. The NVIDIA-ARM AI Research Centre at Cambridge (announced 2024) brings together NVIDIA’s GPU and inference software expertise with ARM’s processor design capabilities, creating a globally significant concentration of edge AI research aligned with healthcare, autonomous vehicles, and robotics application domains.
• The University of Cambridge Department of Engineering hosts research in neural network compression, TinyML, and embedded AI through the Machine Learning group, with connections to the Cambridge Centre for AI in Medicine (CCAIM) and the Epiabm epidemiological modelling consortium. The Cambridge–NVIDIA collaboration specifically targets healthcare edge AI applications, including on-device early disease detection and remote patient monitoring that align with NHS priorities for care-at-home and hospital-at-home pathways. The Alan Turing Institute coordinates national AI research including edge AI deployment for environmental monitoring, smart infrastructure, and Digital Twin applications through its partnership with UKRI and national programme grants.
• The NHS AI Deployment Platform Pilot (NHS England, 2024-2025) explores Edge AI deployment for automated clinical decision support in radiology, ECG analysis, and deterioration early warning — requiring certified edge inference hardware that meets NHS DSP Toolkit data governance standards. The Wellcome Genome Campus in Hinxton, Cambridgeshire, hosts a connected autonomous vehicle corridor feasibility study under the £150m CAM Pathfinder programme, providing a UK test bed for automotive-grade Edge AI System deployment with NHS data and connectivity links. Imperial College London’s AI research in digital health includes wearable biosensor models and on-device ECG classification networks — contributing to the UK’s edge AI healthcare pipeline.
• In Northern England, the Advanced Manufacturing Research Centre (AMRC) at the University of Sheffield operates collaborative industry-academic programmes in edge AI for advanced manufacturing, including vibration-based predictive maintenance and quality control vision systems deployed on production lines at aerospace and automotive suppliers across South Yorkshire and the wider North. The National Robotarium at Heriot-Watt University and University of Edinburgh (Edinburgh, 2022) applies edge AI to robot perception and safe human-robot interaction in manufacturing and healthcare settings. The University of Manchester’s Trustworthy Autonomous Systems Hub (EPSRC, 2021-2026) addresses safety, reliability, and explainability of edge-deployed AI systems, including certification frameworks for Edge AI in regulated environments (medical devices, automotive, aviation). Leeds’s data science and digital health ecosystem, including Nexus at the University of Leeds, connects Edge AI research to NHS West Yorkshire ICB’s digital health agenda.

Future Directions (2026-2030)

• The most significant near-term trajectory is the convergence of Edge AI with small language models (SLMs), enabling LLM-class reasoning capabilities on consumer and industrial edge devices without cloud connectivity. Phi-3-mini and Llama 3.2 running on Qualcomm Snapdragon NPUs represent 2025 capabilities; by 2028, 7-13B parameter models quantised to INT4/INT2 are projected to run on laptop and industrial-grade edge hardware at interactive response rates, enabling genuinely intelligent on-device assistants, code execution environments, and autonomous agents operating without internet dependency.
• Neuromorphic computing (Intel Loihi 2, IBM NorthPole, BrainScaleS) represents a longer-horizon architectural shift: spiking neural networks running on neuromorphic hardware achieve event-driven sparse computation, consuming orders of magnitude less energy than synchronous NPU inference for sensory data with high spatial or temporal sparsity (DVS event cameras, sparse LiDAR returns, intermittent biomedical signals). Full production deployment of neuromorphic Edge AI for real-world sensors is a 2027-2030 timeframe prediction, contingent on progress in spiking neural network training methods and hardware reliability.
• Agentic Edge AI — where edge devices host not just inference models but reasoning agents capable of multi-step planning, tool use, and autonomous decision-making — is emerging from the convergence of SLMs, on-device RAG (retrieval-augmented generation using local knowledge bases), and physical sensor-actuator integration. By 2028, industrial Edge AI Systems are predicted to support agentic maintenance engineers that can diagnose, plan, and execute repair workflows autonomously, drawing on equipment documentation retrieved locally and sensor readings processed on-device without cloud connectivity. Federated Learning will evolve toward personalised federation with differential privacy guarantees, enabling fleets of heterogeneous edge devices to collaboratively maintain accurate, privacy-preserving models that adapt to local distribution shifts without centralising sensitive data.
• The integration of Edge AI with Digital Twin architectures will deepen: edge devices will serve not only as inference endpoints but as the data sources and control actuators for continuously calibrated digital twins, with the twin’s simulation layer running at a local gateway and the optimisation layer in the cloud, enabling real-time what-if analysis and autonomous optimisation at the edge-cloud boundary. Standards for Edge AI System safety certification (IEC 62443 for industrial cybersecurity, ISO 21448 SOTIF for automotive AI systems, IEC 60601 for medical device AI) are being developed by ISO, IEC, and ETSI, with UK BSI participation, creating a regulatory framework that will govern Edge AI deployment in safety-critical applications through to 2030.

Formal Technical Treatment

• The formal specification of an Edge AI System involves quantifying the constraints that distinguish it from unconstrained cloud inference. Let M be a neural network model with parameter count |θ|, memory footprint mem(M) (bytes of weights + activations required for one forward pass), and inference latency lat(M, h) on hardware h. Edge deployment requires: mem(M) ≤ C_mem(h) where C_mem(h) is the available SRAM or shared memory of the target device; lat(M, h) ≤ T_latency where T_latency is the real-time deadline (e.g. 33ms for 30fps video, 1ms for industrial interlocks); and power(M, h) ≤ P_budget where P_budget is the device’s thermal design power (TDP) budget. Quantisation maps each weight w ∈ FP32 to an integer representation w_q = round(w / Δ) where Δ = (w_max - w_min) / (2^b - 1) is the scale factor and b is the bit-width (8 for INT8, 4 for INT4). The memory reduction is approximately a factor of 32/b. The quantisation error for a layer is ε_q = ||W - W_q||_F, and the model accuracy after quantisation degrades by Δacc(b) which empirically is small (< 1%) for INT8 and significant (1-4%) for INT4 without QAT. Quantisation-aware training (QAT) minimises L(θ; D) + λ_q Σ_l ||W_l - Q_b(W_l)||_F^2 where Q_b is the quantisation operator, inserting “fake quantisation” nodes during forward pass to allow the optimiser to minimise the accuracy loss at the target bit-width. Structured pruning removes entire filters F_k from a convolutional layer: a filter importance score S(F_k) = ||F_k||_1 or ||F_k||2 is computed; filters with S(F_k) < τ are zeroed and removed, reducing the layer from C_out to C_out’ output channels. The resulting architecture is then fine-tuned to recover accuracy. The compression ratio r = C_out’/C_out and the FLOPs reduction is approximately r^2 (since both input and output channels shrink in subsequent layers). Knowledge Distillation trains a student S with parameter count |θ_S| << |θ_T| (teacher T) to minimise L_KD = α L_CE(σ(z_S/T), y) + (1-α) L_KD(σ(z_S/T), σ(z_T/T)) where z_S and z_T are student and teacher logit vectors, σ is the softmax function, T is a temperature parameter (T > 1 softens the teacher distribution, revealing inter-class similarity), and y is the one-hot ground truth label; empirically, α ≈ 0.1 and T ≈ 4-8 work well across benchmarks. Federated learning with FedAvg executes T rounds of: each client k ∈ S_t (a random subset of clients) runs E local SGD steps on its local data D_k to produce Δθ_k = θ - θ_k; the server aggregates θ{t+1} = Σ_k (|D_k|/|D|) Δθ_k where |D| = Σ_k |D_k|. Differential privacy adds Gaussian noise N(0, σ^2 C^2 I) to each clipped gradient update (||Δθ_k||_2 ≤ C), achieving (ε, δ)-DP with σ ≈ C√(2 log(1.25/δ))/ε per McMahan et al. (2018). The trade-off between privacy budget ε and model utility (accuracy after federation) is the fundamental privacy-utility curve of federated edge AI systems: lower ε (stronger privacy) requires larger noise σ and more communication rounds to converge to acceptable accuracy.

Benchmark Datasets and Performance Standards

• The MLPerf Tiny benchmark suite (Banbury et al., MLSys 2021) provides the canonical performance evaluation framework for Edge AI Systems, defining four workloads representative of common edge deployment scenarios: (1) Visual Wake Words — binary person-detection from 96x96 grayscale camera frames using MobileNetV1 variants, targeting microcontroller deployment; (2) Image Classification — 10-class CIFAR-10 classification at 32x32 resolution, evaluating throughput and energy on constrained hardware; (3) Keyword Spotting — 12-class audio keyword detection from 1-second MFCC features, representative of always-on voice trigger deployment; (4) Anomaly Detection — multivariate time-series anomaly detection from the ToyADMOS/DCASE industrial machine sound dataset, targeting predictive maintenance applications. Performance metrics in MLPerf Tiny include inference latency (ms), energy per inference (mJ), accuracy on the validation set, and model memory footprint (bytes). For larger-scale edge platforms (Jetson-class, NPU SoCs), MLPerf Edge (formerly MLPerf Inference Edge) evaluates Closed Division (standardised models) and Open Division (any model architecture achieving the accuracy target) across: ImageNet image classification (ResNet-50, MobileNetV4), COCO object detection (SSD-MobileNetV2, YOLOv8), BERT question answering, and 3D-UNet medical image segmentation. The accuracy targets (99% of FP32 reference accuracy for INT8 quantised models, 99.9% for FP32 models) define the quality floor that quantised edge models must meet, preventing vendors from gaming energy metrics with under-performing models. The EEMBC ULPMark-MLPerf benchmark extends evaluation to ultra-low-power MCU inference, measuring inference per joule (inferences/J) on sub-milliwatt class hardware. HAR (Human Activity Recognition) datasets — UCI HAR, PAMAP2, OPPORTUNITY — serve as standard benchmarks for wearable and IMU-based edge AI, evaluating accelerometer-based gesture recognition models. The Edge AI and Vision Alliance’s Edge AI Benchmarks provide vendor-neutral evaluation of NPU and DSP accelerators across a broader workload set than MLPerf. For federated learning benchmarks, LEAF (Caldas et al., 2018) provides non-IID datasets simulating realistic federated edge conditions: FEMNIST (federated handwriting recognition with per-user character partitioning), Shakespeare (federated next-word prediction with per-character text partitioning), and Sentiment140 (per-user Twitter sentiment classification), each exhibiting the statistical heterogeneity that distinguishes real federated edge deployments from IID centralised training.

Standards and Governance

• Edge AI Systems operate within a growing regulatory and standards landscape that spans hardware certification, software quality, data privacy, safety assurance, and sector-specific regulation. At the hardware level, functional safety standards govern safety-critical edge AI deployments: ISO 26262 (Automotive Safety Integrity Level, ASIL) defines requirements for automotive AI systems at ASIL-B through ASIL-D levels, covering design methods, verification, and failure mode analysis for edge perception and control systems in vehicles; IEC 61508 (Functional Safety of Electrical/Electronic/Programmable Electronic Safety-related Systems) and its sector derivatives (IEC 62061 for machinery, IEC 61511 for process industry) apply to industrial Edge AI Systems in safety-critical roles. ISO/PAS 21448 (SOTIF — Safety of the Intended Functionality) specifically addresses AI-specific safety challenges in automotive edge systems: the failure modes arising from insufficiently specified ML models, distribution shift between training and deployment, and sensor degradation that causes incorrect but technically valid model outputs. The EU Machinery Regulation (2023/1230/EU, replacing Machinery Directive 2006/42/EC, applicable from 2027) introduces AI-specific provisions for machine edge AI systems, requiring risk assessments of ML decision-making in safety-related machine control functions. Medical device Edge AI falls under EU MDR (Medical Device Regulation 2017/745) and UK MDR 2002 (amended 2019), which classify diagnostic and therapeutic AI software by risk class (class I through III), with higher classes requiring Notified Body review of the clinical evidence, post-market surveillance, and AI-specific quality management documentation (IEC 62304 for medical software, ISO 14971 for risk management). The FDA in the US issued its “Artificial Intelligence and Machine Learning-Based Software as a Medical Device” guidance (2021, updated 2023) requiring predetermined change control plans (PCCPs) for continuously learning edge AI medical devices, addressing the regulatory challenge of models that update on-device after deployment. For IoT security, ETSI EN 303 645 (Cyber Security for Consumer IoT, implemented in UK PSTI Act 2022, enforced April 2024) mandates baseline security requirements for consumer Edge AI devices: no default universal passwords, managed vulnerability disclosure, secure firmware update mechanisms, secure storage of sensitive parameters, and minimal attack surface — with civil enforcement powers for non-compliant products placed on the UK market. The IEEE P2510 (Standard for Establishing Quality of Data Sensor Parameters for Evaluating AI Edge Devices) and IEEE P2894 (Guide for Ethical Considerations in Artificial Intelligence) represent standards body engagement with edge AI quality and ethics. The EU AI Act (Regulation EU 2024/1689, entered into force August 2024 with phased implementation to 2026) classifies certain edge AI applications as high-risk (biometric identification, emotion recognition in workplaces and schools, critical infrastructure safety, law enforcement facial recognition, medical diagnosis) requiring mandatory conformity assessment, technical documentation including training data description and accuracy metrics, post-market monitoring plans, and human oversight mechanisms. UK AI Governance post-Brexit follows a sector-led, principles-based approach through the AI Safety Institute (AISI) and sector regulators (FCA, CQC, CAA), rather than a horizontal EU AI Act-style regulation, creating divergence that affects UK edge AI product compliance pathways. The National Edge AI Hub (UK, 2024) is working with EPSRC, UKRI, and industry partners to develop test and certification frameworks for edge AI deployments in UK national infrastructure verticals (energy, transport, healthcare), aiming to establish UK-specific certification pathways as a Brexit opportunity in regulatory competitiveness.

Contrast with Related Paradigms

• Edge AI vs. Cloud AI inference: Cloud AI inference centralises model execution on high-throughput GPU servers (NVIDIA A100/H100 clusters), enabling unlimited model size and compute but incurring round-trip network latency (typically 30-200ms for internet-connected devices), requiring continuous connectivity, transmitting sensitive raw data to external servers, and generating significant bandwidth costs. Edge AI inverts these trade-offs: near-zero inference latency (microseconds to milliseconds), offline operation capability, no raw data egress, and minimal bandwidth consumption, at the cost of constrained model capacity and the engineering complexity of compression and on-device deployment. The optimal trade-off depends on the application: safety-critical real-time control (autonomous vehicles, industrial safety interlocks, medical device monitoring) mandates edge inference; non-latency-sensitive batch analytics (monthly transaction fraud review, overnight medical imaging reads) is well-served by cloud inference. Edge-Cloud Collaboration architectures occupy the middle ground, dynamically routing decisions across the hierarchy.
• Edge AI vs. Fog Computing: Fog computing (Bonomi et al., Cisco, 2012) places compute at intermediate network nodes — routers, base stations, edge servers — between end-devices and the cloud. Fog computing is a superset of edge computing: edge computing places compute at or on the device; fog computing places compute at any point in the continuum from device to cloud. Edge AI Systems that run on fog servers (multi-tenant edge inference servers at the network edge, e.g., AWS Wavelength, Azure Edge Zones, NVIDIA EGX) offer higher compute capacity than device-embedded inference (enabling larger models) at the cost of slightly higher latency (typically 5-20ms to a local edge server) and exposure of raw data to the fog node infrastructure. MEC (Multi-access Edge Computing, ETSI MEC) standardises compute offloading from mobile devices to 5G base station-collocated servers, enabling edge AI for mobile applications without on-device inference constraints.
• Edge AI vs. TinyML: TinyML is the subset of Edge AI targeting the most resource-constrained devices — microcontrollers with kilobyte-scale RAM and flash, no operating system, power budgets under 1mW. TinyML represents the extreme end of the edge AI spectrum: while an NVIDIA Jetson Orin running a 25M-parameter YOLO model at 30fps is clearly Edge AI, a 6KB keyword-spotting CNN running on an ARM Cortex-M0 at 1μW is TinyML within Edge AI. The distinction matters for toolchains (TFLM vs. full TFLite), model architectures (micronet vs. MobileNet), and application domains (always-on sensing vs. frame-rate vision). Both are unified by the core requirement of deploying Machine Learning inference within non-cloud hardware constraints, but TinyML adds extreme quantisation (binary, ternary networks), architecture specialisation (MCUNet, sparse MLPs), and memory layout optimisation (scratch buffer reuse, in-place activation computation) beyond standard edge AI practice.

Key Terminology

• Neural Processing Unit (NPU): dedicated silicon block on a SoC that executes matrix multiply-accumulate (MAC) operations with high throughput and energy efficiency for deep learning inference; examples include Apple Neural Engine, Qualcomm Hexagon NPU, ARM Ethos.
• Quantisation: reduction of neural network weight and activation numerical precision from FP32 to INT8, INT4, or lower bit-widths, reducing model memory footprint and inference latency with controlled accuracy degradation.
• Pruning: removal of redundant weights, filters, or attention heads from a trained neural network to reduce model size and computational load; structured pruning removes entire channels (GPU-friendly); unstructured pruning removes individual weights (requires sparse kernels for speedup).
• Knowledge distillation: training a compact student network to mimic the output distribution of a larger teacher network, using teacher soft-label outputs (logit vectors) as training targets to transfer knowledge beyond what hard ground-truth labels provide.
• TinyML: the subdiscipline of Edge AI targeting deployment on microcontrollers with kilobyte-scale RAM budgets (typically <512 KB), sub-milliwatt power budgets, and no operating system; runs on ARM Cortex-M, RISC-V, and Xtensa architectures.
• Federated learning: a distributed machine learning paradigm in which multiple edge devices collaboratively train a shared model by exchanging encrypted gradient updates rather than raw data, preserving data privacy while enabling model improvement from distributed data.
• Differential privacy: a formal mathematical privacy guarantee ensuring that the inclusion or exclusion of any single individual’s data changes the model or output distribution by at most a controlled amount (parameterised by (ε, δ)), achieved by adding calibrated random noise to gradient updates.
• Model compression: the umbrella term for techniques (quantisation, pruning, distillation, NAS) that reduce a neural network’s resource footprint (memory, compute, energy) for deployment on constrained hardware.
• Embedded AI frameworks: software runtime libraries (TensorFlow Lite, ONNX Runtime, ExecuTorch, TFLM, TensorRT) that load, optimise, and execute compressed neural network models on heterogeneous edge hardware, abstracting over CPU, GPU, NPU, and DSP execution backends.
• Hardware-aware NAS (Neural Architecture Search): automated optimisation of neural network topology subject to hardware-specific constraints (memory, latency, energy), producing architectures that are Pareto-optimal for a specific target device rather than for cloud hardware.
• Edge-Cloud Collaboration: tiered inference architecture in which easy, high-confidence decisions are executed locally on the device, while ambiguous or complex inputs are escalated to more capable cloud or fog servers, dynamically balancing latency, accuracy, and bandwidth cost.
• On-device learning: adaptation of model parameters directly on the edge device using locally collected data, without transmitting that data to external servers; includes fine-tuning, continual learning, and reinforcement learning from local reward signals.
• Quantisation-aware training (QAT): training procedure that simulates quantisation effects during the forward pass using “fake quantisation” operators, allowing the model to learn representations robust to the precision reduction applied at deployment; produces more accurate quantised models than post-training quantisation, especially at aggressive bit-widths (INT4 or below).
• TOPS (Tera Operations Per Second): standard measure of neural network inference throughput for NPUs and AI accelerators, measuring the number of multiply-accumulate (MAC) operations per second at INT8 precision; a 20 TOPS NPU executes 20 × 10^12 INT8 MAC operations per second.
• MEC (Multi-access Edge Computing): ETSI standard architecture for deploying compute and AI inference at the 5G base station level, enabling mobile edge AI with 5-20ms latency to a localised inference server.

Research & Literature

• [1] Howard, A.G. et al. (2017). MobileNets: Efficient convolutional neural networks for mobile vision applications. arXiv:1704.04861.
• [2] Sandler, M. et al. (2018). MobileNetV2: Inverted residuals and linear bottlenecks. CVPR 2018.
• [3] Tan, M. and Le, Q.V. (2019). EfficientNet: Rethinking model scaling for convolutional neural networks. ICML 2019.
• [4] Hinton, G., Vinyals, O. and Dean, J. (2015). Distilling the knowledge in a neural network. NIPS Deep Learning Workshop 2014 / arXiv:1503.02531.
• [5] McMahan, B. et al. (2017). Communication-efficient learning of deep networks from decentralized data. AISTATS 2017. [FedAvg — Federated Learning]
• [6] Abadi, M. et al. (2016). Deep learning with differential privacy. ACM CCS 2016. [DP-SGD]
• [7] Frankle, J. and Carlin, M. (2019). The Lottery Ticket Hypothesis: Finding sparse, trainable neural networks. ICLR 2019.
• [8] Lin, J. et al. (2020). MCUNet: Tiny deep learning on IoT devices. NeurIPS 2020. [MIT TinyML]
• [9] Warden, P. and Situnayake, D. (2019). TinyML: Machine Learning with TensorFlow Lite on Arduino and Ultra-Low-Power Microcontrollers. O’Reilly Media.
• [10] Bonawitz, K. et al. (2017). Practical secure aggregation for federated learning on user-held data. arXiv:1611.04482. [Secure aggregation]
• [11] Nagel, M. et al. (2021). A white paper on neural network quantization. Qualcomm AI Research / arXiv:2106.08295.
• [12] Gholami, A. et al. (2021). A survey of quantization methods for efficient neural network inference. arXiv:2103.13630. [UC Berkeley]
• [13] Tishby, N. and Zaslavsky, N. (2015). Deep learning and the information bottleneck principle. IEEE ITW 2015.
• [14] Yang, T.J. et al. (2018). NetAdapt: Platform-aware neural network adaptation for mobile applications. ECCV 2018. [MIT hardware-aware NAS]
• [15] Banbury, C. et al. (2021). MLPerf Tiny: A benchmark suite for ultra-low-power machine learning. MLSys 2021.
• [16] Li, T. et al. (2020). Federated optimization in heterogeneous networks (FedProx). MLSys 2020.
• [17] Shi, W. et al. (2016). Edge computing: Vision and challenges. IEEE Internet of Things Journal, 3(5), 637-646. [Edge computing foundations]
• [18] Mattson, P. et al. (2020). MLPerf Training Benchmark. MLSys 2020.
• [19] Krishnamoorthi, R. (2018). Quantizing deep convolutional networks for efficient inference: A whitepaper. Google / arXiv:1806.08342. [Google quantisation]
• [20] Benchmarking Edge AI Platforms for High-Performance ML Inference (2024). arXiv:2409.14803. [NPU vs GPU latency benchmarks]
• [21] Empowering Edge Intelligence: A Comprehensive Survey on On-Device AI Models (2025). ACM Computing Surveys. DOI:10.1145/3724420.
• [22] Howard, A. et al. (2019). Searching for MobileNetV3. ICCV 2019.
• [23] Chen, Y.H. et al. (2019). Eyeriss v2: A flexible accelerator for emerging deep neural networks on mobile devices. IEEE JETCAS.
• [24] ARM Holdings (2026). Arm at Embedded World 2026: Powering intelligent edge AI systems at scale. ARM Newsroom. https://newsroom.arm.com/blog/arm-embedded-world-2026
• [25] NVIDIA and ARM (2024). NVIDIA and ARM to create AI research centre in Cambridge. NVIDIA Blog. https://blogs.nvidia.com/blog/arm-ai-research-center-cambridge-uk/
• [26] Edge AI Hardware Market Size and Forecast 2025-2034 (2025). Global Market Insights. https://www.gminsights.com/industry-analysis/edge-ai-hardware-market
• [27] Cognitive Edge Computing: A Comprehensive Survey (2025). arXiv:2501.03265.
• [28] National Edge AI Hub — Inside the Hub (2024). STL Partners Report. https://stlpartners.com/articles/edge-computing/inside-the-national-edge-ai-hub/

Provenance