Time-Aware Hybrid Encoding (THE)

• THE is a family of encoding strategies that replaces discrete positions with continuous timestamps to capture irregular temporal dynamics.
• It fuses explicit and implicit memory pathways via hybridization of temporal embeddings, reducing conditional entropy and enhancing prediction accuracy.
• Applications span medical sequence modeling, IoT intrusion detection, and video coding, offering efficient buffering and improved rate-distortion trade-offs.

Time-Aware Hybrid Encoding (THE) denotes a family of encoding strategies and architectural principles designed for neural sequence modeling under nonuniform or irregular temporal dynamics. In THE, temporal information is incorporated as a principal axis of the representation, using hybridization—across either encoding functions or memory paths—to achieve better performance, interpretability, and generalizability than strictly position-based or purely content-driven schemes. THE methodologies have been instantiated in diverse domains, including electronic health record (EHR) modeling, IoT intrusion detection, sequence transmission with variable-length coding, and learned video coding. The hybridization may occur between different families of continuous-time (or timestamp-based) embeddings, or between explicit and implicit temporal memory paths in recurrent architectures.

1. Formal Definitions and Key Variants

THE is characterized by replacing discrete, uniformly-spaced “positions” with real-valued temporal markers (timestamps, elapsed intervals, or normalized intervals), and using these as the argument to one or more time-aware encoding families. Core variants documented in the literature are:

• Continuous-Time Sinusoidal Embedding: For normalized elapsed time $e_t \in [0,1]$, a $d$-dimensional embedding

$$\mathrm{TE}(e_t) = [\phi_1(e_t), \phi_2(e_t), \dots, \phi_d(e_t)]^\top$$

where

$\phi_i(e_t) = \begin{cases} \sin\!\left(e_t/\lambda^{2i/d}\right) & \text{$i$ even}, \ \cos\!\left(e_t/\lambda^{2i/d}\right) & \text{$i$ odd} \end{cases}$

with scaling constant $\lambda$ and dimension $d$ (Jafari et al., 9 Mar 2026, Panopoulos et al., 23 Apr 2026).

• Time-Aware Fourier Embedding: Learnable frequencies $f_j$ modulate the timestamp via

$$\mathrm{PE}_{\text{Fourier}}(t_i)_{2j} = \sin(2\pi f_j t_i), \quad \mathrm{PE}_{\text{Fourier}}(t_i)_{2j+1} = \cos(2\pi f_j t_i)$$

allowing adaptation to domain-specific periodicities (Panopoulos et al., 23 Apr 2026).

• Rotary Positional Encoding (RoPE) with Timestamps: Timestamp $t_i$ modulates the phase $\theta_{i,j}$ in rotational encoding

$d$0

with rotation

$d$1

for each query/key attention slice (Panopoulos et al., 23 Apr 2026).

• Hybrid Memory Pathways: Propagation of temporal context occurs along both explicit (output-recurrence, i.e., decoded frames or visited states) and implicit (hidden-to-hidden, i.e., learned feature states) memory channels. These dual paths are fused at each step, resulting in lower conditional entropy and improved rate-distortion or prediction accuracy (Chen et al., 4 Aug 2025).

2. Implementation Across Domains

THE has found application in medicine, network security, wireless communication, and video compression. Key instantiations are:

Clinical Sequence Modeling

In TA-RNN-Medical-Hybrid, continuous-time sinusoidal embeddings are added to knowledge-graph-enriched visit representations. Explicit time encoding operates independently of visit index, capturing the irregularity of EHR sequences. Embeddings are concatenated with biomedical BERT and SNOMED graph features, remaining fixed during training. The model integrates hierarchical dual-level attention—attending to both visit and concept significance. This hybridization achieves state-of-the-art mortality risk prediction and interpretable, temporally-decomposed risk attribution (Jafari et al., 9 Mar 2026).

IoT Intrusion Detection

A-THENA deploys THE to aggregate fine-grained temporal and packet content information. For a packet flow $d$2 and timestamps $d$3, three encoding families (sinusoidal, Fourier, RoPE) are evaluated, each treating $d$4 as a continuous position. Optimization over the encoding family is data-driven via Early Detection Loss (EDL) in cross-validation. This scheme outperforms index-based and prior temporal encodings, achieving near-zero false alarms and minimal latency on resource-constrained devices (Panopoulos et al., 23 Apr 2026).

Predictive Video Coding

HyTIP implements THE as a hybrid buffering strategy: explicit decoded frames and motion flows are propagated (output-recurrence path), while a compact set of learned features is concurrently buffered (hidden-to-hidden path). Both are merged for motion and inter-frame predictions, minimizing conditional entropy and enhancing coding efficiency at modest buffer sizes. This reconciles the inefficiency of purely output- or hidden-based schemes (Chen et al., 4 Aug 2025).

Variable-Length Transmission Coding

While not labeled THE, variable-length coding with hybrid ARQ employs explicit time-awareness—the encoding/decoding policy and IR packet sizing are optimized to minimize Age-of-Information (AoI) under nonzero feedback/processing delay, explicitly leveraging time in both codebook design and hybrid incremental redundancy (Arafa et al., 2021).

3. Architectural and Algorithmic Principles

Encoder Design

• For time-aware Transformers (e.g., A-THENA), THE is realized through the sum or fusion of time-embedded vectors (input-level) or time-modulated rotations (attention-level), according to the chosen encoding (Panopoulos et al., 23 Apr 2026).
• In RNN-based architectures (e.g., TA-RNN-Medical-Hybrid, HyTIP), the temporal embedding or hybrid propagation is integrated via additive augmentation of input vectors, hybrid memory fusion, or dual-attention mechanisms (Jafari et al., 9 Mar 2026, Chen et al., 4 Aug 2025).
• Model selection over encoding family (in the presence of hybrid variants) is performed via cross-validated loss minimization, ensuring the instantiation best suited to the domain's temporal dynamics (Panopoulos et al., 23 Apr 2026).

Memory and Buffering

Scheme	Explicit Path	Implicit Path	Buffer Size (relative)	Rate-Distortion (BD-rate)
Output-Recurrence Only	Decoded frames	None	High	Baseline
Hidden-to-Hidden Only	None	Many feature maps	Highest	∼–6.5% vs. hybrid
Hybrid (THE/HyTIP)	Decoded frames/flow	Few feature maps	∼14% state-of-art	–21.5% vs. explicit only

Empirical results show that hybrid buffering achieves the lowest bit-rate for a given distortion, with considerably smaller buffer requirements (Chen et al., 4 Aug 2025).

4. Training, Optimization, and Hyperparameters

THE-enabled models are trained using objectives sensitive to temporal classification or rate-distortion trade-offs:

• Loss functions: Recall-oriented F$d$5 loss (TA-RNN-Medical-Hybrid), Early Detection Loss (EDL, A-THENA), joint $d$6 rate-distortion (HyTIP) (Jafari et al., 9 Mar 2026, Panopoulos et al., 23 Apr 2026, Chen et al., 4 Aug 2025).
• Optimization: Adam optimizer, dropout, $d$7 weight decay, and early stopping are typical choices. Encoding-specific hyperparameters such as frequency schedules (Fourier, RoPE) may be fixed or learned depending on the implementation (Jafari et al., 9 Mar 2026, Panopoulos et al., 23 Apr 2026).
• Encoding family selection: Data-driven via validation loss minimization, leveraging multiple variants during model development (Panopoulos et al., 23 Apr 2026).

5. Empirical Performance and Comparative Results

THE demonstrates consistently superior empirical results across domains:

• In MIMIC-III ICU data, TA-RNN-Medical-Hybrid achieves Accuracy = 0.91, AUC = 0.82, and F$d$8 = 0.95, outperforming GRU-D, BEHRT, Med-BERT, and TA-RNN. Improvements in AUC and F$d$9 over TA-RNN are statistically significant ($$\mathrm{TE}(e_t) = [\phi_1(e_t), \phi_2(e_t), \dots, \phi_d(e_t)]^\top$$0) (Jafari et al., 9 Mar 2026).
• In IoT intrusion detection, A-THENA’s THE increases accuracy by 6.88 pp over the best positional encoding, 3.69 pp over the leading feature-based model, and 6.17 pp over time-aware alternatives. Early detection with EDL is substantially improved, earliness (E) maintained below 7 packets, and inference latency reduced to 1–2 ms (Panopoulos et al., 23 Apr 2026).
• In predictive video coding, HyTIP with THE achieves a 21.5% average BD-rate gain over explicit-only baselines, outperforms pure implicit approaches by ∼6.5%, and surpasses VTM 17.0 with a fraction of the buffer size(Chen et al., 4 Aug 2025).

6. Generalization and Adaptation

THE is applicable to any domain characterized by irregular event or data sampling:

• For sequence modeling, event-driven logs, medical time series, sensor data, or user activity traces where temporal granularity and variability are non-uniform and may encode critical contextual information. THE generalizes beyond classification to tasks such as early detection, forecasting, and entropy reduction (Panopoulos et al., 23 Apr 2026).
• Adaptation involves substituting “event” features and their timestamps for domain-specific “packets” or “visits,” applying timestamp normalization ($$\mathrm{TE}(e_t) = [\phi_1(e_t), \phi_2(e_t), \dots, \phi_d(e_t)]^\top$$1), and using cross-validated selection among candidate encoding mechanisms (Panopoulos et al., 23 Apr 2026).

7. Theoretical Insights and Design Guidelines

THE is theoretically supported by conditional entropy reduction under hybrid propagation, as

$$\mathrm{TE}(e_t) = [\phi_1(e_t), \phi_2(e_t), \dots, \phi_d(e_t)]^\top$$2

indicating that fusing explicit and implicit temporal context is always at least as informative as either alone (Chen et al., 4 Aug 2025).

General design principles include:

• Always incorporating previous decoded state in explicit form.
• Learning a minimal yet expressive set of latent features for implicit propagation.
• Fusing all forms of context when computing entropy models or attending over temporal windows.
• Optionally empowering the network to select between context strategies adaptively via masking or attention (Chen et al., 4 Aug 2025).

THE consistently bridges the gap between temporal fidelity and computational or buffer efficiency, outperforming uni-modal schemes in prediction, rate-distortion, and early classification benchmarks (Jafari et al., 9 Mar 2026, Panopoulos et al., 23 Apr 2026, Chen et al., 4 Aug 2025).