AirMoE: Realizing Over-the-Air Distributed Mixture-of-Experts Inference at the Wireless Edge

Abstract: Mixture-of-experts (MoE) architectures enable efficient LLM inference at the wireless edge by reducing per-token computation through sparse expert activation. The wireless distributed MoE (WIDE) architecture addresses edge-device resource constraints by distributing computation-intensive experts across devices coordinated by an edge server. However, repeated uploads of high-dimensional expert outputs over orthogonal multiple access create a severe uplink bottleneck. To overcome this limitation, we propose AirMoE, a novel framework leveraging over-the-air computing (AirComp) to realize simultaneous expert-output aggregation via wireless waveform superposition. Integrating AirComp into MoE inference, however, introduces three unique challenges: fast-varying aggregation weights, layer-dependent error sensitivity, and channel-aware expert placement. To address these challenges, we first construct an inference-aware AirMoE error metric to quantify the effect of physical-layer aggregation distortion on end-to-end (E2E) inference accuracy via perturbation-based layer-sensitivity calibration. We then formulate a joint optimization problem to minimize this E2E error and decompose it, without loss of optimality, into a two-timescale framework. At the fast timescale, we derive a globally optimal threshold-based power-control policy that partitions devices into two groups: those achieving exact aggregation-weight alignment, and the remainder transmitting at maximum power. At the slow timescale, we develop an activation- and channel-aware expert placement strategy that assigns more important experts to devices with lower channel-power cost. Extensive experiments using the OLMoE-1B-7B-0924 model on the ARC-Easy benchmark demonstrate that AirMoE significantly outperforms representative baselines in E2E inference accuracy, particularly under strong device heterogeneity.