Lightweight and Generalizable Multi-Sensor Human Activity Recognition via Cascaded Fusion and Style-Augmented Decomposition

Abstract: Wearable Human Activity Recognition (WHAR) is a prominent research area within ubiquitous computing, whose core lies in effectively modeling intra- and inter-sensor spatio-temporal relationships from multi-modal time series data. Existing methods either suffer from high computational complexity due to attention-based fusion or lack robustness to data variations during feature extraction. To address these issues, we propose a lightweight and generalizable framework that retains the core "decomposition-extraction-fusion" paradigm while introducing two key innovations. First, we replace the computationally expensive Attention and Cross-Variable Fusion (CVF) modules with a Cascaded Fusion Block (CFB), which achieves efficient feature interaction without explicit attention weights through the operational process of "compression-recursion-concatenation-fusion". Second, we integrate a MixStyle-based data augmentation module before the Local Temporal Feature Extraction (LTFE) and Global Temporal Aggregation (GTA) stages. By mixing the mean and variance of different samples within a batch and introducing random coefficients to perturb the data distribution, the model's generalization ability is enhanced without altering the core information of the data. The proposed framework maintains sensor-level, variable-level, and channel-level independence during the decomposition phase, and achieves efficient feature fusion and robust feature extraction in subsequent processes. Experiments on two benchmark datasets (Realdisp, Skoda) demonstrate that our model outperforms state-of-the-art methods in both accuracy and macro-F1 score, while reducing computational overhead by more than 30\% compared to attention-based baselines. This work provides a practical solution for WHAR applications on resource-constrained wearable devices.

• The paper introduces a novel multi-sensor HAR framework that combines a Cascaded Fusion Block with Moment-Morph style augmentation to enhance domain generalization.
• It achieves 96.13% accuracy on Realdisp and 97.50% on Skoda while maintaining a lightweight 3.013M parameter count, outperforming state-of-the-art methods.
• The modular design efficiently manages sensor heterogeneity and supports real-time deployment in wearable, low-resource environments.

Lightweight and Generalizable Multi-Sensor Human Activity Recognition via Cascaded Fusion and Style-Augmented Decomposition

Introduction

The proliferation of wearable sensors has catalyzed substantial progress in Human Activity Recognition (HAR), yet generalization and computational efficiency remain primary challenges, particularly with heterogeneous, multi-modal time-series data characteristic of real-world activity monitoring. This paper introduces a multi-sensor HAR framework that simultaneously achieves high generalization and lightweight deployment through the integration of a novel Cascaded Fusion Block (CFB) for efficient feature integration and a MixStyle-based Moment-Morph (MoM) augmentation strategy to mitigate domain shifts.

Methodology

The framework follows a modular design under the "decomposition-extraction-fusion" paradigm, comprising modality-specific feature embedding, local and global temporal modeling, and two major technical innovations: the CFB and the MoM style augmentation.

The front end employs modality-specific convolutional embeddings to preserve sensor and variable independence. The MoM module, inspired by the MixStyle family, is injected before both the Local Temporal Feature Extraction (LTFE) and the global sequence model (a Mamba module), perturbing low-order feature statistics (mean, variance) across batch samples to simulate cross-domain style variations, thus compelling the model to learn style-invariant temporal features and significantly enhancing domain generalization.

The CFB replaces conventional cross-variable fusion (CVF) and attention-based fusion modules. Instead of explicit attention, CFB employs a parameter-efficient "compression–recursion–concatenation–fusion" pipeline. Recursive depthwise convolutions with multi-order receptive fields extract local temporal interactions, which are subsequently concatenated and projected to reconstruct channel-level representations. This structure enables effective multi-scale dynamic modeling with linear computational complexity.

The global backbone includes the Mamba sequence model for high-capacity temporal aggregation, followed by an optional cross-sensor interaction module for modeling inter-sensor dependencies. The classification head is a standard fully connected linear layer.

Figure 1: Model architecture integrating modality-specific embeddings, Moment-Morph style augmentation, CFB-based fusion, and Mamba sequence aggregation.

Experimental Results

Benchmark Evaluation

Extensive experiments were conducted on the Realdisp and Skoda WHAR benchmarks. The proposed model demonstrated consistent superiority in both accuracy and macro-F1 score against eight state-of-the-art baselines. On Realdisp, the model achieved 96.13% accuracy and 95.30% macro-F1, providing an absolute improvement of 3.55% and 3.37% over DynamicWHAR, the highest-performing baseline. On Skoda, it attained 97.50% accuracy and 97.14% macro-F1, exceeding HARMamba by 3.54% and 5.27%, respectively. These results unequivocally validate the claim that hybridization of CFB and MoM yields synergistic generalization and efficiency gains.

Figure 2: Experimental comparison reveals superior accuracy and macro-F1 across both Realdisp and Skoda benchmarks.

Efficiency

The model's compactness is evident by its parameter count (3.013M), which is less than half of attention-based models (e.g., 7.354M for Att. Model) and less than most competitive lightweight architectures. Inference speed on a resource-constrained ARM Cortex-A53 reaches 8.8 ms/sample, compared to 11.6 ms for DynamicWHAR and 38.1 ms for Att. Model. This efficiency is directly attributed to the elimination of attention mechanisms in favor of the CFB, leading to a >30% reduction in computational cost.

Figure 3: Parameter count and inference latency demonstrate the architecture's suitability for deployment on low-power wearable devices.

Ablation and Module Analysis

Ablation studies reinforce the contribution of each module:

• Exclusion of both MoM and CFB yields only 89.2% accuracy, with substantial improvements observed when adding either module.
• The CFB alone reduces model size and inference time while boosting accuracy (+3.57%) and macro-F1 (+3.75%).
• The combination of MoM and CFB realizes the highest scores, confirming that moment-based style augmentation and efficient fusion operate orthogonally.

Implications and Future Directions

The framework is well-positioned for deployment in resource-limited, real-time WHAR scenarios typical of clinical monitoring, elderly care, and sports analytics. By decoupling fusion efficiency from reliance on expensive attention mechanisms and providing systematic generalization under domain shift, the model reduces the overhead typically associated with state-of-the-art deep HAR methods.

Theoretically, the results underscore the efficacy of distribution-level feature augmentation (via MoM/MixStyle) when combined with scalable, multi-scale fusion blocks. This paradigm offers new opportunities for robust, transferable temporal modeling in broader time-series or multi-sensor learning tasks.

Future research directions include extending evaluation to additional datasets capturing greater environmental and sensor diversity, injecting adaptive MixStyle strategies deeper or in more flexible locations in the network, and exploring the integration of self-supervised or semi-supervised learning to further reduce label dependency. Additionally, the CFB module’s general applicability for fusion in other domains (e.g., multimodal video or physiological time series) warrants further investigation.

Conclusion

This work presents a multi-sensor HAR model that combines a novel Cascaded Fusion Block with a MixStyle-based Moment-Morph augmentation module to outperform strong baselines in both recognition accuracy and computational efficiency. The approach achieves robust generalization across subjects and devices while reducing parameter count and inference latency, facilitating practical deployment in wearable and low-resource systems. The modular fusion strategy and style-based augmentation offer promising avenues for generalized sequence modeling under domain shift, with implications extending far beyond WHAR.