---
title: 'CNN-HAR: CNNs for Human Activity Recognition'
url: https://www.emergentmind.com/topics/cnn-har
type: topic
---

# CNN-HAR: CNNs for Human Activity Recognition

Searching arXiv for the cited papers to ground the article and confirm identifiers.
CNN-HAR denotes the use of convolutional neural networks for human activity recognition, more broadly for recognizing human behavior from structured measurements such as wearable inertial streams, smartphone sensors, radar micro-Doppler maps, WiFi spectrograms, CSI traces, depth sequences, and video clips. In the survey literature, CNN-based HAR is organized into four major input categories—multimodal sensing devices, smartphones, radar, and vision devices—and is typically described as a pipeline comprising sensing, data collection or transmission, CNN-based feature extraction and classification, and activity inference or notification [2202.03274]. The abbreviation is not completely uniform across the literature: closely related work also uses HAR for human action recognition in video and, in a different subfield, human attribute recognition from images [2305.19896]; [1911.11351].

## 1. Scope and problem formulation

At its core, CNN-HAR replaces handcrafted feature engineering with convolutional representation learning over raw or lightly transformed signals. The central claim recurring across benchmark, survey, and application papers is that CNNs are effective because they learn discriminative local patterns, hierarchical structure, and cross-channel dependencies directly from input windows or image-like encodings, reducing dependence on manually designed descriptors [2501.08471]; [2202.03274].

The term covers several distinct problem settings. In inertial HAR, the input is usually a multivariate time series from accelerometers, gyroscopes, magnetometers, or heart-rate sensors. In radar and WiFi sensing, the inputs are micro-Doppler signatures, CSI-derived traces, or spectrograms. In depth or video-based action recognition, the input is a sequence of frames or a 3D spatio-temporal tensor. A concise taxonomy is given below.

| Setting | Typical representation | Representative papers |
|---|---|---|
| Wearable and smartphone HAR | Raw windows, FFT/PSD features, CWT scalograms | [1906.01935], [2101.06709], [2106.12666] |
| RF sensing HAR | Micro-Doppler maps, WiFi spectrograms, CSI wavelet features | [2510.22772], [2606.02974], [2212.13161] |
| Vision and depth HAR | Sequential images, signal images, 3D CNN clips | [2010.16073], [2303.17218], [2305.19896] |

A persistent methodological theme is that classification accuracy is only one dimension of the problem. Several recent systems explicitly target edge deployment, low latency, energy efficiency, memory reduction, or FPGA realizability, indicating that CNN-HAR has evolved from an accuracy-only benchmark exercise into a co-design problem spanning signal representation, model structure, and hardware mapping [2510.22772]; [2102.01875]; [2305.12824].

## 2. Input representations and preprocessing regimes

CNN-HAR does not rely on a single canonical input format. One important branch feeds raw inertial windows directly to the network after segmentation. In a lower-limb IMU study, each sensor window is arranged as a \(6 \times 204\) matrix, and multiple sensors are stacked as channels to form an input of shape \(6 \times 204 \times N\), where rows are IMU components, columns are time samples, and channels correspond to sensor placements [1906.01935]. This representation is explicitly described as a model-driven input adaptation rather than handcrafted feature extraction.

A second branch transforms one-dimensional signals into frequency or time-frequency images before convolution. The multichannel CNN on UCI HAR constructs two complementary views—FFT-based frequency features and Welch power-spectrum features—and processes them in parallel CNN channels before fusion [2101.06709]. The CWT-based pipeline on UniMiB SHAR converts accelerometer signals into scalogram images, compares multiple mother wavelets, and reports that Mexican Hat is the strongest single wavelet while a combination of second- and fourth-order wavelets yields the best reported accuracy of \(99.26\%\) [2106.12666]. This suggests that, in CNN-HAR, preprocessing is often not a peripheral step but part of the representational design.

Multimodal depth-inertial HAR uses yet another image-conversion strategy. MGAF converts depth sequences into Sequential Front View Images and six inertial streams into Signal Images using a specific stacking order, then applies twin CNNs followed by multistage fusion [2010.16073]. WiFi HAR in WISE-HAR treats spectrograms as RGB images resized to \(224 \times 224 \times 3\) because the pre-trained CNN backbones expect three-channel input [2606.02974]. Radar HAR in Neural-HAR uses STFT-preprocessed \((1,30,28)\) micro-Doppler frames from the UoG2020 continuous radar dataset [2510.22772].

CSI-based HAR adds a different preprocessing logic centered on redundancy suppression. PCWCNN first applies PCA-based subcarrier fusion, discards the first principal component, retains PC2 and PC3, uses Savitzky–Golay smoothing, performs adaptive segmentation, and then applies a three-level DWT before the Wavelet CNN classifier [2212.13161]. The paper states that PCA reduces an original 90-subcarrier sample to two principal components while maintaining strong recognition performance. A plausible implication is that CNN-HAR pipelines often benefit when signal compression is aligned with sensing physics rather than delegated entirely to the CNN.

## 3. Core architectural patterns

The simplest CNN-HAR models retain a conventional CNN stack. A representative example uses three convolutional layers, three max-pooling layers, three fully connected layers, ReLU activations, dropout with keep probability \(0.5\), Adam, cross-entropy loss, and a softmax output layer [1906.01935]. In the broad five-dataset benchmark over UCI-HAR, OPPORTUNITY, PAMAP2, WISDM, and Berkeley MHAD, CNNs consistently delivered the strongest or near-strongest performance among the deep models, with particularly strong results on OPPORTUNITY, PAMAP2, and Berkeley MHAD [2501.08471].

Beyond standard stacks, CNN-HAR frequently uses multi-branch or multi-channel layouts. The two-channel UCI-HAR model processes frequency and power features in parallel and concatenates the channel outputs before final classification [2101.06709]. FieldHAR adopts a branch-wise feature-fusion CNN for heterogeneous sensors, with one branch per modality, three convolution layers per branch, global max pooling, concatenation of branch outputs, and two dense layers, and reports that feature fusion outperforms data fusion while using far fewer trainable parameters: \(89.13\%\) accuracy with \(2{,}900\) trainable parameters versus \(85.43\%\) with \(71{,}756\) for data fusion [2305.12824].

Dynamic parameterization is another major pattern. In CondConv-HAR, standard convolution is replaced by an input-dependent mixture of expert kernels:
\[
\mathrm{Output}=\sigma\left(\left(\alpha_1 W_1+\alpha_2 W_2+\dots+\alpha_n W_n\right)*X\right),
\]
with routing weights computed from a global-average-pooled summary of the input [2006.03259]. The reported effect is a substantial increase in model capacity with only a modest increase in FLOPs. On WISDM, accuracy rises from \(98.12\%\) for \(n=1\) to \(99.60\%\) for \(n=8\), while FLOPs rise from \(30.01\)M to \(31.69\)M; on OPPORTUNITY, the best result reaches \(81.18\%\) [2006.03259].

Residual and 3D convolutional architectures extend CNN-HAR into deeper temporal modeling. DanHAR augments a CNN or residual-CNN backbone with channel and temporal attention and reports that the attention-based residual network achieves \(98.85\%\) on WISDM, \(79.03\%\) on UniMiB SHAR, \(93.16\%\) on PAMAP2, and \(82.75\%\) on OPPORTUNITY with negligible parameter overhead [2006.14435]. For video-based human action recognition, toolflow papers such as HARFLOW3D and fpgaHART target 3D CNNs including C3D, Slowonly, R(2+1)D, and X3D, emphasizing that temporal depth improves action understanding but sharply increases compute and memory requirements [2303.17218]; [2305.19896].

## 4. Fusion, attention, and gating

A substantial part of CNN-HAR research concerns how to fuse modalities, scales, and temporal contexts without overwhelming the classifier with redundant or weakly informative channels. The HUF model addresses multi-sensor IMU fusion through three cascaded CNN autoencoder blocks: a Data Representation Stacked Autoencoder for axis-level short-time features, a Local Feature Fusion AE for six-channel sensor-unit fusion, and a Global Feature Fusion AE for cross-sensor fusion [2502.04489]. The best configuration uses eight convolutional layers in each AE, an overcomplete first-stage code with 256 kernels, and a final global code of 64; the reported accuracies are \(97\%\) on DaLiAc, \(97\%\) on UCI-HAR, and \(88\%\) on the Parkinson’s disease gait dataset [2502.04489].

Attention mechanisms in CNN-HAR typically separate the questions of modality salience and temporal salience. DanHAR formalizes this explicitly as “what to focus on” and “where to focus,” using channel attention to weight sensor modalities or feature channels and temporal attention to weight time regions inside the window [2006.14435]. On WISDM, the ablation study reports \(98.09\%\) for channel attention only, \(98.12\%\) for temporal attention only, \(98.62\%\) for temporal-plus-channel, and \(98.85\%\) for channel-plus-temporal, indicating that dual attention is superior and that channel-first ordering is slightly better [2006.14435].

Gating is a closely related but architecturally distinct idea. In MGAF, Gated Average Fusion applies sigmoid gates derived from a high-boost kernel to modality-specific feature maps and fuses them while preserving the dimensionality of a single feature map:
\[
F_f(j)=G_1 \odot F_1(j)+G_2 \odot F_2(j).
\]
The method reports \(99.3\%\) on UTD-MHAD, \(99.85\%\) on Berkeley MHAD, and \(99.8\%\) on UTD Kinect-V2, while using \(1{,}229{,}814\) parameters in the SFI+SI setting versus \(49{,}232{,}121\) in prior multistage fusion baselines [2010.16073]. Neural-HAR introduces a related but sensor-specific mechanism: GateCNN first performs Doppler vector embedding and then uses dual-path gated temporal convolutions, with the central update
\[
\mathbf{Y}=\mathbf{X}_{conv5}\odot \mathrm{ReLU}(\mathbf{Z})+\mathbf{X}_{conv1},
\]
where the residual path stabilizes training [2510.22772].

Ensemble fusion operates at the decision level rather than the feature level. WISE-HAR averages class-probability outputs from Deep CNN, Wide CNN, MobileNetV2, ResNet50V2, and EfficientNetB0 using soft voting, and reports \(94.87\%\) on LOS/Biquad, outperforming the best individual model, MobileNetV2 at \(94.21\%\), by \(0.66\) percentage points [2606.02974]. It also reports small cross-condition drops of \(1.37\%\) for LOS-to-NLOS and \(2.07\%\) for LOS/Biquad-to-LOS/PIFA transfer, a result presented as evidence of strong generalization [2606.02974].

## 5. Edge deployment and accelerator co-design

A defining recent development in CNN-HAR is the shift from benchmark models to deployable systems. AHAR targets microcontroller-class wearable devices using a multi-output 1-D CNN and a decision-tree Output Block Predictor that selects whether a segment should exit after the first output block or continue through the full baseline [2102.01875]. On Opportunity, the baseline and adaptive architectures achieve weighted F1 scores of \(91.79\%\) and \(91.57\%\); on w-HAR, they achieve \(97.55\%\) and \(97.64\%\), respectively. Measured on an EFM32 Giant Gecko microcontroller at 14 MHz, the adaptive model reduces per-segment energy from \(460.41\ \mu J\) to \(403.71\ \mu J\) on Opportunity and from \(488.74\ \mu J\) to \(434.29\ \mu J\) on w-HAR [2102.01875].

Neural-HAR makes hardware co-design even more explicit. GateCNN reaches \(86.4\%\) accuracy on UoG2020 with only \(2.7\)k parameters and \(0.28\)M FLOPs per inference, and its FPGA prototype on a Xilinx Zynq-7000 Z-7007S at 100 MHz achieves \(107.5\ \mu s\) latency, \(9.3\)k inference/s throughput, \(0.104\) W total power, and \(15\) mW dynamic power using LUT-based ROM and distributed RAM only, with zero DSPs and zero BRAMs [2510.22772]. The design rationale is that removing recurrent sequential dependence yields a more deterministic and hardware-friendly datapath than CNN-RNN hybrids.

At the system level, FieldHAR integrates data acquisition, synchronization, windowing, and integer CNN inference in SystemVerilog. For a kitchen activity scenario with heterogeneous sensors, the optimized serial design on an Intel Cyclone IV uses as low as \(25\%\) logic elements and \(2\%\) memory bits of the low-end FPGA, with less than \(1\%\) accuracy loss from the FP32 offline model, \(0.54\) ms serial latency, and \(1{,}851\) labels/s throughput [2305.12824]. This reinforces the point that in edge HAR, sensor interfacing and synchronous acquisition can be as critical as neural inference.

For video-based HAR, accelerator toolflows generalize the same co-design logic to 3D CNNs. HARFLOW3D is latency-oriented and reports ablation gains of \(1.14\times\) from combination or separation of computation nodes, \(1.52\times\) from activation fusion, and \(18.21\times\) from runtime reconfiguration of layer parameters; it also states that it can achieve up to \(5\times\) better performance than some existing works [2303.17218]. fpgaHART is throughput-oriented and reports, on ZCU102 with batch size 100, \(13.44\) clips/s and \(85.96\) GOp/s for X3D, while supporting modern 3D models beyond older C3D-centric designs [2305.19896].

## 6. Empirical regimes, misconceptions, and open problems

Reported CNN-HAR performance is strongly dataset- and protocol-dependent, and direct numerical comparison across papers is often invalid. The survey literature explicitly warns that datasets, splits, sensing modalities, and evaluation protocols differ substantially [2202.03274]. This caution is reflected in the literature itself: CWT-CNN work reports \(99.26\%\) on UniMiB SHAR [2106.12666], the multichannel FFT/PSD CNN reports \(95.25\%\) on UCI HAR [2101.06709], CondConv reports \(94.01\%\) on PAMAP2 and \(99.60\%\) on WISDM [2006.03259], and Neural-HAR reports \(86.4\%\) on continuous radar UoG2020 [2510.22772]. These are not interchangeable performance statements but results under different sensing assumptions and task definitions.

Several recurrent confusions recur across modalities. Sitting versus standing remains difficult in UCI-HAR-style problems [2101.06709]; similar static postures such as sitting, standing, and lying remain the main errors in hierarchical IMU fusion [2502.04489]; walking-related confusions appear in UCI-HAR and OPPORTUNITY benchmarks [2501.08471]. This suggests that CNN-HAR is often limited less by the absence of deep representation learning than by intrinsic overlap in short-window sensor evidence.

One common misconception is that CNN-HAR is synonymous with raw-signal 1-D convolution. The literature shows a much broader reality: transformed inputs such as CWT scalograms, SI/SFI image encodings, spectrograms, wavelet coefficients, and PCA-compressed CSI are central to many of the strongest pipelines [2106.12666]; [2010.16073]; [2212.13161]. Another misconception is that higher recognition accuracy automatically implies practical superiority. Edge-oriented work repeatedly shows that recurrent bottlenecks, parameter memory, off-chip traffic, and synchronization overhead can dominate deployment feasibility even when accuracy differences are small [2510.22772]; [2102.01875].

The principal open problems identified in review work are data scarcity and class imbalance, lack of multi-person and group-activity datasets, limited contextual information, poor dataset standardization and metadata quality, reproducibility gaps, sensor and subject variability, computational cost on edge devices, real-world versus controlled-environment mismatch, and privacy [2202.03274]. Future directions proposed in that review include generative models for underrepresented activities, future activity prediction, context-aware HAR, standardized dataset metadata, more realistic datasets, and efficient deployable architectures [2202.03274]. A plausible synthesis of the recent literature is that the field is moving toward modality-aware representation design, structured fusion, and hardware-conscious CNN construction rather than ever-larger generic backbones alone.

Source: https://www.emergentmind.com/topics/cnn-har