Modality-Aware Zero-Shot Pruning and Sparse Attention for Efficient Multimodal Edge Inference

Abstract: Edge devices increasingly run multimodal sensing pipelines that must remain accurate despite fluctuating power budgets and unpredictable sensor dropout. Existing pruning methods fail under these conditions: they generally require fine-tuning after compression, consuming over $10\times$ the deployment energy, and they assign static importance scores that are blind to which sensors are present. We present the SentryFuse framework, which addresses both challenges jointly through two key components. First, SentryGate learns modality-conditioned importance scores during training via first-order saliency supervision and then prunes attention heads and feed-forward channels at deployment without fine-tuning. Second, SentryAttend replaces dense self-attention, a key bottleneck in contemporary multimodal architectures, with sparse grouped-query attention, yielding a net 15% reduction in GFLOPs across three different multimodal architectures. Across three applications and multimodal backbones, SentryGate achieves a 12.7% average accuracy improvement over the strongest pruning baseline, and upto to 18% under modality dropout conditions. Together, SentryFuse reduces memory by 28.2% and lowers latency by up to $1.63\times$ without further fine-tuning, establishing modality-aware zero-shot compression as a practical path to multimodal intelligence on heterogeneous edge hardware.

• The paper introduces Sentry, a modality-aware zero-shot pruning method that computes importance scores in a single forward pass without fine-tuning.
• It replaces dense self-attention with a sparse grouped-query mechanism to reduce compute complexity from quadratic to near-linear while maintaining accuracy.
• Empirical results demonstrate up to 18% accuracy improvement under sensor dropout and a 28.2% reduction in memory footprint on edge platforms.

Modality-Aware Zero-Shot Pruning and Sparse Attention for Efficient Multimodal Edge Inference

Introduction and Motivation

Efficient deployment of multimodal time-series models on edge hardware is challenged by the energy and compute constraints typical of IoT devices and the prevalence of sensor dropout. Classical pruning pipelines and transformer backbones assumed static modality presence and relied on compute-heavy fine-tuning to maintain accuracy post-compression. This results in excessive deployment costs and suboptimal performance under modality missingness, as static saliency assignment and dense self-attention mechanisms fail to robustly adapt to dynamic sensor sets and limited power budgets.

The Sentry framework introduces a principled solution to these critical obstacles through two tightly integrated advancements: (1) modality-aware zero-shot pruning of neural structural units (attention heads, feed-forward neurons, and MoE experts), and (2) a sparse grouped-query attention mechanism as an efficient drop-in attention replacement. This dual approach addresses both practical system deployment constraints and theoretical questions on cross-modal saliency allocation.

Figure 1: Multimodal and multivariate modeling of heterogeneous sensor streams under missingness, demonstrating robustness of multimodal architectures compared to classical baselines.

Modality-Aware Structural Pruning: Sentry

Existing structured pruning methods assign static importance scores to model units, presuming a fixed sensor input distribution. As demonstrated empirically (Figure 2), such static pruning leads to severe performance degradation, especially under random modality dropout, since attention heads and neurons may have differential importance for different sensor configurations.

Figure 2: The superiority of Sentry's modality-aware pruning over the modality-agnostic SynFlow baseline under diverse missingness regimes.

Sentry utilizes a lightweight modality-conditioned gating module. During training, Sentry learns a mapping from the modality-availability mask to an importance score for each model unit, supervised by a first-order Taylor saliency alignment objective. The backbone is exposed to a curriculum of increasing modality dropout, ensuring the learned gating function is robust to arbitrary missingness patterns. Crucially, Sentry is optimized such that at deployment, importance scores are computed by a single forward-pass evaluation, and subnetworks are instantiated in a zero-shot manner without further fine-tuning or gradient computations, which is infeasible on edge platforms (Figure 3).

Figure 3: Energy comparison reveals over 10× higher consumption for fine-tuning-based pruning versus Sentry's zero-shot approach, with negligible accuracy loss.

Figure 4: Sentry's training and inference workflow, demonstrating simultaneous structural importance learning and grouped-query attention for practical edge deployment.

Sparse Grouped-Query Attention

Dense self-attention is profile-dominant in both memory and latency on embedded platforms (Figure 5). Building upon redundancy in temporal attention, Sentry replaces dense multi-head attention with grouped-query attention (GQA), wherein key-value projections are shared among head groups, reducing parameter count and compute. This is further augmented by token-level sparsity (retaining only the top-$U$ salient queries per group), reducing temporal complexity from quadratic to near-linear in sequence length, while preserving accuracy.

Figure 5: Self-attention dominates compute time in transformer-based multimodal architectures; compression must target this bottleneck for practical gains.

Figure 6: Empirical attention long-tail statistics highlight the redundancy motivating sparse selection.

Experimental Results

Extensive evaluations on standard multimodal time-series datasets (WESAD, DSADS, DaliaHAR) and heterogeneous architectures (MAESTRO, FlexMoE, FuseMoE) confirm that Sentry consistently outperforms baseline structured pruning methods by an average of 12.7% in predictive accuracy, with improvements reaching up to 18% under severe modality missingness (Figure 7). Sentry is also competitive with strong gradient-based saliency teachers, recovering 95–98% of their accuracy while eliminating deployment-time gradient computation.

Figure 7: Accuracy resilience of Sentry-equipped models (MAESTRO, FlexMoE, FuseMoE) with increasing numbers of missing modalities.

Ablative studies demonstrate that Sentry's performance is robust to the number of GQA groups and can be tuned for various trade-offs between parameter sharing and expressiveness (Figure 8).

Figure 8: Accuracy under different numbers of grouped-query attention groups and modality dropout, illustrating Sentry's flexibility.

Sentry delivers a 28.2% reduction in memory footprint and up to 1.63× reduction in end-to-end latency without sacrificing predictive accuracy or requiring post-pruning fine-tuning. When used with post-training quantization (FP32, FP16, FP8), Sentry models lie on or near the accuracy-storage Pareto frontier (Figure 9), demonstrating compatibility with classical compression techniques. Hardware benchmarks on GPU, CPU, Jetson TX2, iPhone 13 Pro, and Google Pixel 8 show that accuracy gains and efficiency improvements hold consistently across platforms.

Figure 9: Pareto trade-off between accuracy and storage under Sentry pruning and quantization.

Figure 10: Latency, power, and total energy consumption for Sentry under different quantization and pruning regimes.

Practical and Theoretical Implications

Sentry establishes that modality-aware, zero-shot pruning is both feasible and necessary for multimodal models deployed in the presence of missing sensors and variable power. Theoretical implications include the emergence of smooth cross-modal saliency curves and a pragmatic separation of saliency assignment from task optimization. By learning a forward-pass predictor of importance, Sentry amortizes the compute cost of saliency estimation over training and decouples deployment from any need for on-device fine-tuning. This methodology reifies the "train once, deploy many" paradigm, which is critical for real-time mobile intelligence and federated inference scenarios.

On the efficiency axis, dense attention computation is replaced with a parameter- and compute-efficient mechanism that preserves the statistical utility of temporal cross-modal dependencies while achieving substantial real-world latency and energy reductions.

Figure 11: Heatmaps of attention weight distributions during the pruning lifecycle, highlighting the progressive concentration of representational capacity.

Future Prospects

The Sentry framework opens several new directions in energy-constrained multimodal learning and neural architecture optimization:

• Joint search over pruning and quantization policies for multimodal models with end-to-end trainable deployment cost objectives.
• Extension to more complex dynamic environments with adversarial or context-dependent sensor failures.
• Adapting Sentry for continual learning and on-the-fly retraining as new modalities are added or removed.
• Incorporation into real-time control and robot perception stacks where latency and energy constraints are paramount and sensor availability is nonstationary.

Conclusion

Sentry presents a rigorous and practical approach for edge deployment of multimodal time-series models by unifying robust, modality-aware zero-shot pruning with a sparse, efficient attention design. The framework is empirically validated across a spectrum of architectures, applications, datasets, and platforms, exhibiting consistent advantages in accuracy, computational efficiency, and energy consumption without recourse to post-hoc fine-tuning. Sentry thus shifts the paradigm for deploying multimodal intelligence, rendering it adaptable to sensor failures and the stringent requirements of next-generation embedded hardware.

Figure 12: Comparative pruning accuracy on DaliaHAR: Sentry closely tracks a strong Taylor-based saliency teacher, substantially outperforming random pruning at all sparsity levels.