CNN-HAR: CNNs for Human Activity Recognition
- CNN-HAR is a method that applies CNNs to extract features directly from raw or lightly transformed multimodal sensor signals for human activity recognition.
- It replaces handcrafted feature engineering with hierarchical convolutional representation learning that captures local patterns and cross-channel dependencies.
- Recent studies integrate fusion techniques, attention mechanisms, and hardware co-design to improve accuracy, efficiency, and edge deployment feasibility.
Searching arXiv for the cited papers to ground the article and confirm identifiers. CNN-HAR denotes the use of convolutional neural networks for human activity recognition, more broadly for recognizing human behavior from structured measurements such as wearable inertial streams, smartphone sensors, radar micro-Doppler maps, WiFi spectrograms, CSI traces, depth sequences, and video clips. In the survey literature, CNN-based HAR is organized into four major input categories—multimodal sensing devices, smartphones, radar, and vision devices—and is typically described as a pipeline comprising sensing, data collection or transmission, CNN-based feature extraction and classification, and activity inference or notification (Islam et al., 2022). The abbreviation is not completely uniform across the literature: closely related work also uses HAR for human action recognition in video and, in a different subfield, human attribute recognition from images (Toupas et al., 2023); (Wu et al., 2019).
1. Scope and problem formulation
At its core, CNN-HAR replaces handcrafted feature engineering with convolutional representation learning over raw or lightly transformed signals. The central claim recurring across benchmark, survey, and application papers is that CNNs are effective because they learn discriminative local patterns, hierarchical structure, and cross-channel dependencies directly from input windows or image-like encodings, reducing dependence on manually designed descriptors (Hossain et al., 14 Jan 2025); (Islam et al., 2022).
The term covers several distinct problem settings. In inertial HAR, the input is usually a multivariate time series from accelerometers, gyroscopes, magnetometers, or heart-rate sensors. In radar and WiFi sensing, the inputs are micro-Doppler signatures, CSI-derived traces, or spectrograms. In depth or video-based action recognition, the input is a sequence of frames or a 3D spatio-temporal tensor. A concise taxonomy is given below.
| Setting | Typical representation | Representative papers |
|---|---|---|
| Wearable and smartphone HAR | Raw windows, FFT/PSD features, CWT scalograms | (Bevilacqua et al., 2019, Sikder et al., 2021, Nedorubova et al., 2021) |
| RF sensing HAR | Micro-Doppler maps, WiFi spectrograms, CSI wavelet features | (Wu et al., 26 Oct 2025, Arshad et al., 2 Jun 2026, Showmik et al., 2022) |
| Vision and depth HAR | Sequential images, signal images, 3D CNN clips | (Ahmad et al., 2020, Toupas et al., 2023, Toupas et al., 2023) |
A persistent methodological theme is that classification accuracy is only one dimension of the problem. Several recent systems explicitly target edge deployment, low latency, energy efficiency, memory reduction, or FPGA realizability, indicating that CNN-HAR has evolved from an accuracy-only benchmark exercise into a co-design problem spanning signal representation, model structure, and hardware mapping (Wu et al., 26 Oct 2025); (Rashid et al., 2021); (Liu et al., 2023).
2. Input representations and preprocessing regimes
CNN-HAR does not rely on a single canonical input format. One important branch feeds raw inertial windows directly to the network after segmentation. In a lower-limb IMU study, each sensor window is arranged as a matrix, and multiple sensors are stacked as channels to form an input of shape , where rows are IMU components, columns are time samples, and channels correspond to sensor placements (Bevilacqua et al., 2019). This representation is explicitly described as a model-driven input adaptation rather than handcrafted feature extraction.
A second branch transforms one-dimensional signals into frequency or time-frequency images before convolution. The multichannel CNN on UCI HAR constructs two complementary views—FFT-based frequency features and Welch power-spectrum features—and processes them in parallel CNN channels before fusion (Sikder et al., 2021). The CWT-based pipeline on UniMiB SHAR converts accelerometer signals into scalogram images, compares multiple mother wavelets, and reports that Mexican Hat is the strongest single wavelet while a combination of second- and fourth-order wavelets yields the best reported accuracy of (Nedorubova et al., 2021). This suggests that, in CNN-HAR, preprocessing is often not a peripheral step but part of the representational design.
Multimodal depth-inertial HAR uses yet another image-conversion strategy. MGAF converts depth sequences into Sequential Front View Images and six inertial streams into Signal Images using a specific stacking order, then applies twin CNNs followed by multistage fusion (Ahmad et al., 2020). WiFi HAR in WISE-HAR treats spectrograms as RGB images resized to because the pre-trained CNN backbones expect three-channel input (Arshad et al., 2 Jun 2026). Radar HAR in Neural-HAR uses STFT-preprocessed micro-Doppler frames from the UoG2020 continuous radar dataset (Wu et al., 26 Oct 2025).
CSI-based HAR adds a different preprocessing logic centered on redundancy suppression. PCWCNN first applies PCA-based subcarrier fusion, discards the first principal component, retains PC2 and PC3, uses Savitzky–Golay smoothing, performs adaptive segmentation, and then applies a three-level DWT before the Wavelet CNN classifier (Showmik et al., 2022). The paper states that PCA reduces an original 90-subcarrier sample to two principal components while maintaining strong recognition performance. A plausible implication is that CNN-HAR pipelines often benefit when signal compression is aligned with sensing physics rather than delegated entirely to the CNN.
3. Core architectural patterns
The simplest CNN-HAR models retain a conventional CNN stack. A representative example uses three convolutional layers, three max-pooling layers, three fully connected layers, ReLU activations, dropout with keep probability $0.5$, Adam, cross-entropy loss, and a softmax output layer (Bevilacqua et al., 2019). In the broad five-dataset benchmark over UCI-HAR, OPPORTUNITY, PAMAP2, WISDM, and Berkeley MHAD, CNNs consistently delivered the strongest or near-strongest performance among the deep models, with particularly strong results on OPPORTUNITY, PAMAP2, and Berkeley MHAD (Hossain et al., 14 Jan 2025).
Beyond standard stacks, CNN-HAR frequently uses multi-branch or multi-channel layouts. The two-channel UCI-HAR model processes frequency and power features in parallel and concatenates the channel outputs before final classification (Sikder et al., 2021). FieldHAR adopts a branch-wise feature-fusion CNN for heterogeneous sensors, with one branch per modality, three convolution layers per branch, global max pooling, concatenation of branch outputs, and two dense layers, and reports that feature fusion outperforms data fusion while using far fewer trainable parameters: accuracy with trainable parameters versus with for data fusion (Liu et al., 2023).
Dynamic parameterization is another major pattern. In CondConv-HAR, standard convolution is replaced by an input-dependent mixture of expert kernels: 0 with routing weights computed from a global-average-pooled summary of the input (Cheng et al., 2020). The reported effect is a substantial increase in model capacity with only a modest increase in FLOPs. On WISDM, accuracy rises from 1 for 2 to 3 for 4, while FLOPs rise from 5M to 6M; on OPPORTUNITY, the best result reaches 7 (Cheng et al., 2020).
Residual and 3D convolutional architectures extend CNN-HAR into deeper temporal modeling. DanHAR augments a CNN or residual-CNN backbone with channel and temporal attention and reports that the attention-based residual network achieves 8 on WISDM, 9 on UniMiB SHAR, 0 on PAMAP2, and 1 on OPPORTUNITY with negligible parameter overhead (Gao et al., 2020). For video-based human action recognition, toolflow papers such as HARFLOW3D and fpgaHART target 3D CNNs including C3D, Slowonly, R(2+1)D, and X3D, emphasizing that temporal depth improves action understanding but sharply increases compute and memory requirements (Toupas et al., 2023); (Toupas et al., 2023).
4. Fusion, attention, and gating
A substantial part of CNN-HAR research concerns how to fuse modalities, scales, and temporal contexts without overwhelming the classifier with redundant or weakly informative channels. The HUF model addresses multi-sensor IMU fusion through three cascaded CNN autoencoder blocks: a Data Representation Stacked Autoencoder for axis-level short-time features, a Local Feature Fusion AE for six-channel sensor-unit fusion, and a Global Feature Fusion AE for cross-sensor fusion (Arabzadeh et al., 6 Feb 2025). The best configuration uses eight convolutional layers in each AE, an overcomplete first-stage code with 256 kernels, and a final global code of 64; the reported accuracies are 2 on DaLiAc, 3 on UCI-HAR, and 4 on the Parkinson’s disease gait dataset (Arabzadeh et al., 6 Feb 2025).
Attention mechanisms in CNN-HAR typically separate the questions of modality salience and temporal salience. DanHAR formalizes this explicitly as “what to focus on” and “where to focus,” using channel attention to weight sensor modalities or feature channels and temporal attention to weight time regions inside the window (Gao et al., 2020). On WISDM, the ablation study reports 5 for channel attention only, 6 for temporal attention only, 7 for temporal-plus-channel, and 8 for channel-plus-temporal, indicating that dual attention is superior and that channel-first ordering is slightly better (Gao et al., 2020).
Gating is a closely related but architecturally distinct idea. In MGAF, Gated Average Fusion applies sigmoid gates derived from a high-boost kernel to modality-specific feature maps and fuses them while preserving the dimensionality of a single feature map: 9 The method reports 0 on UTD-MHAD, 1 on Berkeley MHAD, and 2 on UTD Kinect-V2, while using 3 parameters in the SFI+SI setting versus 4 in prior multistage fusion baselines (Ahmad et al., 2020). Neural-HAR introduces a related but sensor-specific mechanism: GateCNN first performs Doppler vector embedding and then uses dual-path gated temporal convolutions, with the central update
5
where the residual path stabilizes training (Wu et al., 26 Oct 2025).
Ensemble fusion operates at the decision level rather than the feature level. WISE-HAR averages class-probability outputs from Deep CNN, Wide CNN, MobileNetV2, ResNet50V2, and EfficientNetB0 using soft voting, and reports 6 on LOS/Biquad, outperforming the best individual model, MobileNetV2 at 7, by 8 percentage points (Arshad et al., 2 Jun 2026). It also reports small cross-condition drops of 9 for LOS-to-NLOS and 0 for LOS/Biquad-to-LOS/PIFA transfer, a result presented as evidence of strong generalization (Arshad et al., 2 Jun 2026).
5. Edge deployment and accelerator co-design
A defining recent development in CNN-HAR is the shift from benchmark models to deployable systems. AHAR targets microcontroller-class wearable devices using a multi-output 1-D CNN and a decision-tree Output Block Predictor that selects whether a segment should exit after the first output block or continue through the full baseline (Rashid et al., 2021). On Opportunity, the baseline and adaptive architectures achieve weighted F1 scores of 1 and 2; on w-HAR, they achieve 3 and 4, respectively. Measured on an EFM32 Giant Gecko microcontroller at 14 MHz, the adaptive model reduces per-segment energy from 5 to 6 on Opportunity and from 7 to 8 on w-HAR (Rashid et al., 2021).
Neural-HAR makes hardware co-design even more explicit. GateCNN reaches 9 accuracy on UoG2020 with only $0.5$0k parameters and $0.5$1M FLOPs per inference, and its FPGA prototype on a Xilinx Zynq-7000 Z-7007S at 100 MHz achieves $0.5$2 latency, $0.5$3k inference/s throughput, $0.5$4 W total power, and $0.5$5 mW dynamic power using LUT-based ROM and distributed RAM only, with zero DSPs and zero BRAMs (Wu et al., 26 Oct 2025). The design rationale is that removing recurrent sequential dependence yields a more deterministic and hardware-friendly datapath than CNN-RNN hybrids.
At the system level, FieldHAR integrates data acquisition, synchronization, windowing, and integer CNN inference in SystemVerilog. For a kitchen activity scenario with heterogeneous sensors, the optimized serial design on an Intel Cyclone IV uses as low as $0.5$6 logic elements and $0.5$7 memory bits of the low-end FPGA, with less than $0.5$8 accuracy loss from the FP32 offline model, $0.5$9 ms serial latency, and 0 labels/s throughput (Liu et al., 2023). This reinforces the point that in edge HAR, sensor interfacing and synchronous acquisition can be as critical as neural inference.
For video-based HAR, accelerator toolflows generalize the same co-design logic to 3D CNNs. HARFLOW3D is latency-oriented and reports ablation gains of 1 from combination or separation of computation nodes, 2 from activation fusion, and 3 from runtime reconfiguration of layer parameters; it also states that it can achieve up to 4 better performance than some existing works (Toupas et al., 2023). fpgaHART is throughput-oriented and reports, on ZCU102 with batch size 100, 5 clips/s and 6 GOp/s for X3D, while supporting modern 3D models beyond older C3D-centric designs (Toupas et al., 2023).
6. Empirical regimes, misconceptions, and open problems
Reported CNN-HAR performance is strongly dataset- and protocol-dependent, and direct numerical comparison across papers is often invalid. The survey literature explicitly warns that datasets, splits, sensing modalities, and evaluation protocols differ substantially (Islam et al., 2022). This caution is reflected in the literature itself: CWT-CNN work reports 7 on UniMiB SHAR (Nedorubova et al., 2021), the multichannel FFT/PSD CNN reports 8 on UCI HAR (Sikder et al., 2021), CondConv reports 9 on PAMAP2 and 0 on WISDM (Cheng et al., 2020), and Neural-HAR reports 1 on continuous radar UoG2020 (Wu et al., 26 Oct 2025). These are not interchangeable performance statements but results under different sensing assumptions and task definitions.
Several recurrent confusions recur across modalities. Sitting versus standing remains difficult in UCI-HAR-style problems (Sikder et al., 2021); similar static postures such as sitting, standing, and lying remain the main errors in hierarchical IMU fusion (Arabzadeh et al., 6 Feb 2025); walking-related confusions appear in UCI-HAR and OPPORTUNITY benchmarks (Hossain et al., 14 Jan 2025). This suggests that CNN-HAR is often limited less by the absence of deep representation learning than by intrinsic overlap in short-window sensor evidence.
One common misconception is that CNN-HAR is synonymous with raw-signal 1-D convolution. The literature shows a much broader reality: transformed inputs such as CWT scalograms, SI/SFI image encodings, spectrograms, wavelet coefficients, and PCA-compressed CSI are central to many of the strongest pipelines (Nedorubova et al., 2021); (Ahmad et al., 2020); (Showmik et al., 2022). Another misconception is that higher recognition accuracy automatically implies practical superiority. Edge-oriented work repeatedly shows that recurrent bottlenecks, parameter memory, off-chip traffic, and synchronization overhead can dominate deployment feasibility even when accuracy differences are small (Wu et al., 26 Oct 2025); (Rashid et al., 2021).
The principal open problems identified in review work are data scarcity and class imbalance, lack of multi-person and group-activity datasets, limited contextual information, poor dataset standardization and metadata quality, reproducibility gaps, sensor and subject variability, computational cost on edge devices, real-world versus controlled-environment mismatch, and privacy (Islam et al., 2022). Future directions proposed in that review include generative models for underrepresented activities, future activity prediction, context-aware HAR, standardized dataset metadata, more realistic datasets, and efficient deployable architectures (Islam et al., 2022). A plausible synthesis of the recent literature is that the field is moving toward modality-aware representation design, structured fusion, and hardware-conscious CNN construction rather than ever-larger generic backbones alone.