Feature Anchors for Time-Series Sensor-Based Human Activity Recognition

Abstract: Wearable Human Activity Recognition (HAR) still lacks a representation that is both explicit and adaptable. Handcrafted time-series features (TSFs) capture meaningful motion statistics and remain competitive on standard benchmarks, but they are usually used as fixed preprocessing outputs. Deep models learn adaptable representations directly from raw signals, but those representations are typically latent and difficult to inspect. We address this gap by treating handcrafted TSFs as feature anchors: explicit intermediate representations that remain inside the model and are adjusted by neural context instead of being discarded. We propose the Temporal Conditioning Network for Feature Anchors (TCNet), which extracts handcrafted anchors, encodes complementary time-domain and frequency-domain context from raw IMU windows, and predicts context-conditioned scale, bias, and gating parameters to modulate anchor groups directly in feature space. This design keeps anchor semantics visible while allowing the representation to adapt to the classification objective. Across five HAR benchmarks, TCNet achieves 70.2% mF1 on USC-HAD, 85.1% mF1 on Daphnet, 93.9% mF1 on MHealth, and 94.5% mF1 on PAMAP2. Relative to rTsfNet, it improves by 4.5 points on USC-HAD, 14.6 points on Daphnet, and 6.5 points on MHealth. Ablations show that the gains come primarily from anchor guidance rather than simple branch fusion, and feature-space analyses indicate that several discriminative TSF families are not reliably accessible in standard latent representations. These results suggest that, for HAR, handcrafted TSFs are most useful when they remain explicit and adaptable within the model. The code is available at: https://github.com/ni-x-lab/TCNet-har

• The paper introduces TCNet that uses context-conditioned affine modulation to adapt handcrafted time-series features as interpretable anchors.
• It demonstrates significant SOTA performance improvements and parameter efficiency across five diverse HAR benchmarks using explicit TSFs.
• Empirical analysis confirms that dynamic TSF correction enhances class separation while preserving the semantic integrity of key statistical features.

Feature Anchors for Time-Series Sensor-Based Human Activity Recognition: An Authoritative Synthesis

Introduction and Motivation

"Feature Anchors for Time-Series Sensor-Based Human Activity Recognition" (2604.25092) addresses a persistent challenge in wearable HAR: reconciling the explicit, semantically-interpretable representations delivered by handcrafted time-series features (TSFs) with the adaptable, task-optimized latent representations learned by deep nets. While TSFs, such as spectral concentration, crossings, and autocorrelation coefficients, remain competitive baselines for HAR due to their interpretability and compactness, their role has been largely restricted to fixed preprocessing, whereas deep models typically ignore such priors, instead optimizing opaque embeddings from raw signals. This work argues that handcrafted features should be systematically treated as explicit, adaptable feature anchors within the neural model, rather than as disposable preprocessing steps.

The proposed solution, Temporal Conditioning Network for Feature Anchors (TCNet), embeds TSFs as explicit, intermediate anchor representations. Rather than being static, these anchors are adaptively corrected via context-dependent affine modulation (scale and bias), as determined by both time- and frequency-domain embeddings of the raw IMU window. This approach maintains interpretable structure, supports robust recognition across significant inter-subject and temporal variability, and achieves strong numerical efficiency and accuracy margins over large contemporary HAR backbones—while requiring orders of magnitude fewer parameters.

Methodology: The TCNet Architecture

TCNet institutes an explicit architectural separation between handcrafted statistical anchor features and adaptive, context-conditioned correction mechanisms.

After partitioning each IMU window into multi-scale blocks, TCNet:

1. Extracts handcrafted TSFs for each block, spanning diverse families (filterbank, spectral, basic and higher-order statistics, temporal crossings, quantiles, and autocorrelations), resulting in a blockwise anchor tensor.
2. Derives global context embeddings via lightweight temporal and frequency branches:
   • Temporal Context: 1D convs followed by pooling to summarize local transitions.
   • Frequency Context: STFT-based spectrograms aggregated via MLP-Mixer, capturing periodic and spectral cues.
3. Predicts affine corrections for each anchor family in each block—a scale and bias plus a gating coefficient—using concatenated context. Affine transforms are interpolated with the identity to preserve anchor semantics and enable graded adaptation.
4. Aggregates and fuses the corrected views along sensor, temporal, and multi-scale axes for downstream classification.

This formulation is fundamentally distinct from hybrid approaches that append TSFs as auxiliary inputs or side-branches; here, anchors are the core explicit representation, refined but never replaced by neural context.

Figure 1: TCNet adapts handcrafted time-series feature anchors with context-conditioned scale and bias, improving class separability while preserving interpretability.

Key technical design elements include differentiable versions of all TSFs, regularization losses to bound correction magnitude (enforcing conservative adaptation), and hierarchical fusion strategies to maximize information flow without inflating parameter count.

Figure 2: Architecture: parallel computation of context embeddings and explicit TSFs, followed by direct, context-conditioned anchor correction and hierarchical fusion.

Empirical Evaluation

Benchmarks and Baselines

Experiments are conducted on five canonical HAR datasets: USC-HAD, UCI-HAR, Daphnet, MHealth, and PAMAP2, each selected for diversity in sampling, channel count, class cardinality, and domain specificity (e.g., clinical detection in Daphnet).

Baselines span:

• Classical RF pipelines on TSFs.
• Generic sequence/transformer/frequency/time series architectures (DeepConvLSTM, TimesNet, Crossformer, etc.).
• Strong hybrid methods (rTsfNet), and LLM-inspired SensorLLM.

Main Results

TCNet sets new state-of-the-art (SOTA) for macro-F1 on four out of five benchmarks, including the two hardest—Daphnet and MHealth—often with large margins over both RF-TSF (+14.6 points on Daphnet) and heavy neural backbones (for example, outperforming rTsfNet with only 1/20th the parameters).

Figure 3: TCNet dominates accuracy and macro-F1 tradeoff across five HAR benchmarks, surpassing both end-to-end deep nets and RF-TSF reference.

The numerical efficiency is notable. While rTsfNet leverages ~20M parameters, TCNet achieves superior or competitive results with ~1M.

Figure 4: Parameter efficiency: TCNet achieves higher mF1 with far fewer parameters, dominating per-dataset-metric cells against SOTA baselines.

Ablation and Representation Analysis

Ablation studies directly attribute TCNet’s gains to the anchor adaptation mechanism rather than just increased capacity or naive ensemble/fusion. Disabling context-conditioned correction reverts performance to below the fixed RF-TSF baseline—even when extra branches remain.

Figure 5: Performance drops sharply when anchor correction is ablated, confirming adaptation, not mere fusion, is responsible for gains.

Further, visualization of TSF distributions pre- and post-correction shows that TCNet’s interpolated correction tightens or shifts only selected anchor families (e.g., Temporal Crossing), improving class separation without erasing feature semantics.

Figure 6: Corrected TSF distributions reveal selective, interpretable modulation—dynamic activities are sharpened, while core statistics remain largely intact.

Latent Representation Limits and the Case for Explicit Anchors

Linear probes from SOTA deep encoders (e.g., TimesNet) to each TSF family indicate systematic limitations: several highly discriminative TSF families (as measured by RF importance) are not reliably accessible from latent embeddings. In some cases, supervised training actually suppresses linear recoverability of those features.

Figure 7: TSF families with the highest discriminative power (e.g., Spectral, Statistics) are not recovered well in latent space—supervised training sometimes worsens linear accessibility versus random initialization.

This provides strong evidence that, for domains like HAR where well-established statistical summaries are available, reliance on end-to-end latent feature learning may forfeit both interpretability and discriminative power.

Anchor-Aligned Pretraining and Transfer

Compact TCNet, a lightweight anchor-guided encoder, was pretrained using in-domain pretext tasks (Arrow of Time, Permutation, TimeWarp) and compared with large-scale, cross-domain SSL frameworks (UKB-SSL). Despite being trained on two orders of magnitude less data, anchor-aligned pretraining consistently outperformed UKB-SSL and matched or exceeded RF-TSF, particularly under domain shift (e.g., for PAMAP2’s multi-IMU regime).

Figure 8: Compact TCNet, pretrained on minimal in-domain data, matches or surpasses UKB-SSL pretrained on ~100k subjects; anchor-aligned SSL is data-efficient and robust to domain shift.

Linear probes on the frozen UKB-SSL encoder reveal negligible or negative $R^2$ for all TSF families when crossing to the PAMAP2 domain, indicating that large-scale latent pretraining fails to preserve analytic TSF structure under distributional mismatch.

Figure 9: All discriminative TSF families have negative $R^2$ w.r.t. the frozen UKB-SSL encoder under strong domain shift, whereas random initialization retains some structure.

Implications for Human Activity Recognition and Sensor-Based Learning

The results strongly support the thesis that, in structured domains like HAR, the optimal role for handcrafted features is neither mere preprocessing nor auxiliary side input, but as explicit, model-internal anchors subject to context-aware neural correction. TCNet’s hybrid anchor design couples the statistical interpretability and compactness of analytic TSFs with the adaptability and robustness of modern neural networks—delivering superior numerical efficiency, competitive transfer efficiency, and meaningful introspection of intermediate representations.

The findings generalize beyond HAR. Whenever well-founded, hand-engineered feature sets exist (e.g., clinical biomarkers or physics-based signal summaries), embedding them as visible, adaptable anchors may provide an interpretable, efficient, and robust alternative to fully latent end-to-end pipelines.

Limitations and Future Work

Evaluation is confined to IMU-based HAR; extension to physiological signal modalities (e.g., ECG, PPG) will require domain-tailored feature sets. The anchor set is currently fixed; automatic discovery or sparsification of the TSF vocabulary represents an open direction. Finally, while anchor correction improves interpretability, integrated tools for auditing or visualizing anchor trajectories in real-world deployments should be developed.

Conclusions

This work establishes a formal methodology for hybrid explicit-anchor representation learning in HAR. Context-conditioned TSF anchors within TCNet yield SOTA empirical performance, high parameter efficiency, and semantically tractable representations—validating the central claim that explicit, adaptable domain statistics constitute a robust, interpretable foundation for modern sensor activity recognition models (2604.25092). Explicit feature anchors, not opaque deep embeddings, should be the default intermediate representation whenever strong handcrafted priors are available.