Temporal Encoding Module (TEM)

• TEM is a family of mechanisms that transforms continuous or discrete temporal structures into representations used for downstream prediction and control.
• In deep learning, TEMs act as front-end encoders that aggregate sequence history into embeddings, improving tasks like motion prediction and video analysis.
• In signal processing and neuromorphic systems, TEMs operate as Time Encoding Machines, converting analog signals into event times for efficient reconstruction and inference.

Temporal Encoding Module (TEM) denotes a family of mechanisms that convert temporal structure into a representation usable by downstream computation. In contemporary sequence modeling, the term is often used functionally for a front-end block that encodes history into features for prediction; in classical signal processing, spiking neural networks, and event-based sensing, closely related literature uses TEM to mean a Time Encoding Machine, namely a device that converts continuous-time signals into event times rather than uniformly sampled amplitudes. The literature therefore treats TEM both as a learned temporal feature extractor and as an event-driven analog-to-time encoder, with the common theme that temporal information is represented explicitly and then consumed by downstream inference, reconstruction, or control (Lebailly et al., 2020, Adam, 2021, Adam et al., 2022).

1. Terminological scope and conceptual split

The term is not used uniformly across the cited literature. Some papers present a deep-learning component that is functionally equivalent to a temporal encoding module without using that exact name, while others use TEM in the classical signal-processing sense of a Time Encoding Machine.

Usage	Representative role	Representative papers
Learned temporal encoder	Encodes sequence history into embeddings for prediction or matching	(Lebailly et al., 2020, Chen et al., 2023, Hu et al., 10 Apr 2025)
Time Encoding Machine	Encodes analog signals into event times or inter-event intervals	(Adam, 2021, Tarnopolsky et al., 2022, Liu et al., 2023)
Neuromorphic temporal compute primitive	Uses event time, delay, and causal margin as computational variables	(R et al., 2023, &&&10&&&)

A recurrent source of confusion is that several papers explicitly describe TEM-like functionality while stating that they do not define a standalone component literally named “Temporal Encoding Module.” In the spiking paper on time encoding, TEM is a classical signal-processing concept rather than a modern software block (Adam, 2021). In the traffic-prediction paper, temporal encoding is distributed across a spatial-temporal encoding stage, a spatial-temporal encode inferring module, and a temporal dilated causal convolutional network rather than isolated as a single TEM (Hu et al., 10 Apr 2025). This suggests that “TEM” is best understood as a role in a pipeline rather than a single canonical architecture.

At the same time, the role itself is stable. A TEM-like component either transforms observed temporal history into a learned embedding for downstream prediction, or transforms a continuous-time signal into temporally meaningful events whose timing carries the information required for reconstruction or inference. That shared abstraction links motion forecasting, VideoQA, traffic prediction, spiking training, event cameras, and time-based ADCs.

2. Learned temporal encoding modules in sequence models

In learned sequence models, a TEM typically sits at the front end of a larger architecture and encodes temporal history before downstream relational or decision modules operate on it. The clearest example is the Temporal Inception Module (TIM) in human motion prediction, where TIM functions as the temporal encoding module of a two-stage pipeline: temporal encoding with TIM, followed by spatial/joint dependency modeling and residual prediction with a GCN (Lebailly et al., 2020).

In that formulation, the motion input is represented as a collection of $K=3I$ scalar coordinate trajectories,

$$X = [X^0, \cdots, X^k, \cdots, X^{K-1}], \qquad X^k = [X_{-M}^{k}, X_{-M+1}^{k}, \cdots, X_{-1}^{k}],$$

and TIM operates separately on each coordinate trajectory. The module samples nested subsequences $X^k_{-M_j:-1}$, applies multiple 1D temporal convolutions with different kernel sizes at each subsequence scale, concatenates branch outputs within a scale,

$$E_j^k = \text{concat}(C_{S^j_1}(X^k), C_{S^j_2}(X^k), \cdots, C_{S^j_L}(X^k)),$$

and then concatenates across scales,

$E^k = \text{concat}(E_1^k, E_2^k, \cdots, E_J^k).$

This design encodes short-window/local dynamics and long-window/coarse dynamics into a single feature vector that becomes the node feature input to the downstream GCN (Lebailly et al., 2020).

The specific TIM configuration used for benchmark comparison employs two subsequences, $M_1=5$ and $M_2=10$, with kernel banks $(12\times2, 9\times3)$ on length 5 and $(9\times3, 7\times5, 6\times7, 1\times1)$ on length 10. The final per-coordinate embedding size is explicitly given as

$$223 = 12\cdot4 + 9\cdot3 + 9\cdot8 + 7\cdot6 + 6\cdot4 + 1\cdot10.$$

The GCN then predicts residual motion through

$$X = [X^0, \cdots, X^k, \cdots, X^{K-1}], \qquad X^k = [X_{-M}^{k}, X_{-M+1}^{k}, \cdots, X_{-1}^{k}],$$0

The paper reports consistent gains over DCT+GCN on Human3.6M, including $$X = [X^0, \cdots, X^k, \cdots, X^{K-1}], \qquad X^k = [X_{-M}^{k}, X_{-M+1}^{k}, \cdots, X_{-1}^{k}],$$1 vs $$X = [X^0, \cdots, X^k, \cdots, X^{K-1}], \qquad X^k = [X_{-M}^{k}, X_{-M+1}^{k}, \cdots, X_{-1}^{k}],$$2 at $$X = [X^0, \cdots, X^k, \cdots, X^{K-1}], \qquad X^k = [X_{-M}^{k}, X_{-M+1}^{k}, \cdots, X_{-1}^{k}],$$3 ms and $$X = [X^0, \cdots, X^k, \cdots, X^{K-1}], \qquad X^k = [X_{-M}^{k}, X_{-M+1}^{k}, \cdots, X_{-1}^{k}],$$4 vs $$X = [X^0, \cdots, X^k, \cdots, X^{K-1}], \qquad X^k = [X_{-M}^{k}, X_{-M+1}^{k}, \cdots, X_{-1}^{k}],$$5 at $$X = [X^0, \cdots, X^k, \cdots, X^{K-1}], \qquad X^k = [X_{-M}^{k}, X_{-M+1}^{k}, \cdots, X_{-1}^{k}],$$6 ms, which the authors interpret as evidence that learned multi-timescale temporal encoding improves especially long-horizon prediction (Lebailly et al., 2020).

A different deep-learning realization appears in Tem-Adapter for VideoQA, where the TEM-equivalent component is the Temporal Aligner placed on top of frozen CLIP ViT-B/32 frame features (Chen et al., 2023). The encoder refines a sequence of CLIP frame embeddings,

$$X = [X^0, \cdots, X^k, \cdots, X^{K-1}], \qquad X^k = [X_{-M}^{k}, X_{-M+1}^{k}, \cdots, X_{-1}^{k}],$$7

and a training-only autoregressive decoder predicts frame features from history and language guidance,

$$X = [X^0, \cdots, X^k, \cdots, X^{K-1}], \qquad X^k = [X_{-M}^{k}, X_{-M+1}^{k}, \cdots, X_{-1}^{k}],$$8

This is not merely temporal pooling: the decoder uses a causal mask and fused visual-language memory, and the full loss is

$$X = [X^0, \cdots, X^k, \cdots, X^{K-1}], \qquad X^k = [X_{-M}^{k}, X_{-M+1}^{k}, \cdots, X_{-1}^{k}],$$9

The paper reports that removing the Temporal Aligner drops SUTD-TrafficQA from $X^k_{-M_j:-1}$0 to $X^k_{-M_j:-1}$1 and MSR-VTT-MC from $X^k_{-M_j:-1}$2 to $X^k_{-M_j:-1}$3, while removing the autoregressive objective drops them to $X^k_{-M_j:-1}$4 and $X^k_{-M_j:-1}$5, respectively (Chen et al., 2023).

In traffic forecasting, temporal encoding is more distributed. STEI-PCN introduces absolute temporal coordinate embeddings

$X^k_{-M_j:-1}$6

for within-day and day-of-week indices, and relative temporal distance embeddings $X^k_{-M_j:-1}$7 for lag labels $X^k_{-M_j:-1}$8 (Hu et al., 10 Apr 2025). These embeddings feed a dynamic adjacency inference rule,

$X^k_{-M_j:-1}$9

which modulates local spatial-temporal aggregation, while a three-layer temporal dilated causal convolutional network captures longer-range temporal dependencies through

$$E_j^k = \text{concat}(C_{S^j_1}(X^k), C_{S^j_2}(X^k), \cdots, C_{S^j_L}(X^k)),$$0

The temporal absolute and relative encodings contribute modest but measurable gains in ablation, while removing the entire TDCN causes larger degradation, for example on PeMS04 MAE from $$E_j^k = \text{concat}(C_{S^j_1}(X^k), C_{S^j_2}(X^k), \cdots, C_{S^j_L}(X^k)),$$1 to $$E_j^k = \text{concat}(C_{S^j_1}(X^k), C_{S^j_2}(X^k), \cdots, C_{S^j_L}(X^k)),$$2 (Hu et al., 10 Apr 2025).

3. Time Encoding Machines: event times as the representation

In classical TEM literature, a TEM is not a learned embedding block but an event-driven sampler. The canonical instance is the integrate-and-fire TEM, which encodes an analog signal into a sequence of event times by integrating a biased input until a fixed threshold is reached. This idea underlies work on spiking training, event cameras, asynchronous ADCs, and time-domain communication receivers (Tarnopolsky et al., 2022, Adam et al., 2022, Naaman et al., 2023).

For bounded bandlimited signals, a standard IF-TEM law is

$$E_j^k = \text{concat}(C_{S^j_1}(X^k), C_{S^j_2}(X^k), \cdots, C_{S^j_L}(X^k)),$$3

or equivalently, in terms of inter-spike intervals $$E_j^k = \text{concat}(C_{S^j_1}(X^k), C_{S^j_2}(X^k), \cdots, C_{S^j_L}(X^k)),$$4,

$$E_j^k = \text{concat}(C_{S^j_1}(X^k), C_{S^j_2}(X^k), \cdots, C_{S^j_L}(X^k)),$$5

If $$E_j^k = \text{concat}(C_{S^j_1}(X^k), C_{S^j_2}(X^k), \cdots, C_{S^j_L}(X^k)),$$6, then

$$E_j^k = \text{concat}(C_{S^j_1}(X^k), C_{S^j_2}(X^k), \cdots, C_{S^j_L}(X^k)),$$7

and classical recovery theory uses conditions such as

$$E_j^k = \text{concat}(C_{S^j_1}(X^k), C_{S^j_2}(X^k), \cdots, C_{S^j_L}(X^k)),$$8

for $$E_j^k = \text{concat}(C_{S^j_1}(X^k), C_{S^j_2}(X^k), \cdots, C_{S^j_L}(X^k)),$$9-bandlimited signals (Tarnopolsky et al., 2022). In this view, the event times are a sampling representation, and timing precision replaces amplitude precision as the dominant concern.

Several papers extend this formulation. For bandpass signals, a two-channel IF-TEM can be interpreted through periodic nonuniform sampling, with a sufficient reconstruction condition

$E^k = \text{concat}(E_1^k, E_2^k, \cdots, E_J^k).$0

for a single-band bandpass signal of bandwidth $E^k = \text{concat}(E_1^k, E_2^k, \cdots, E_J^k).$1, thereby tying the TEM parameters to bandwidth rather than upper spectral edge (Liu et al., 2023). For FRI signals, IF-TEM sampling is combined with a designed kernel $E^k = \text{concat}(E_1^k, E_2^k, \cdots, E_J^k).$2 so that the filtered signal

$E^k = \text{concat}(E_1^k, E_2^k, \cdots, E_J^k).$3

retains a finite set of Fourier coefficients, and the timing-derived interval measurements become linear in those coefficients (Naaman et al., 2021, Kamath et al., 2021).

Event-camera theory uses the same mathematical abstraction. “How Asynchronous Events Encode Video” models an event-based sensor array as many TEMs indexed by spatial position, each producing event times according to threshold-triggered temporal evolution. In that setting, the paper shows that for bandlimited video, spatial sensor density and temporal resolution become coupled, so spatial oversampling can improve temporal recoverability in a way impossible for frame-based cameras (Adam et al., 2022). This suggests that time encoding is not just sparse sensing, but a nonuniform sampling mechanism over space-time.

The same conceptual move appears in communication receivers. “Symbol Detection Using an Integrate-and-Fire Time Encoding Receiver” uses an IF-TEM front end with threshold-area law

$E^k = \text{concat}(E_1^k, E_2^k, \cdots, E_J^k).$4

and shows that waveform reconstruction is not necessary for symbol detection; the time encodings themselves are sufficient for direct detection, and the paper derives an analytical approximation for symbol error probability that closely matches Monte Carlo simulation (Bernardo, 25 Aug 2025).

4. Learning, reconstruction, and direct inference from temporal encodings

Once temporal information has been encoded, downstream processing varies sharply by domain. In learned temporal encoders, embeddings are passed to GCNs, Transformer heads, or multi-view predictors. In Time Encoding Machine settings, the central tasks are parameter recovery, signal reconstruction, or direct inference from event times.

A prominent example is training SNNs by treating spike times as constraints. The paper on time encoding for SNNs interprets each integrate-and-fire neuron as a time encoder and writes threshold conditions of the form

$E^k = \text{concat}(E_1^k, E_2^k, \cdots, E_J^k).$5

with additional no-earlier-spike inequalities

$E^k = \text{concat}(E_1^k, E_2^k, \cdots, E_J^k).$6

Because spike times are fixed in the training formulation, these equations are linear in the weights, so one-layer SNN training becomes a linear or constrained-linear problem rather than standard backpropagation through a discontinuous spike nonlinearity (Adam, 2021). The paper explicitly frames this as an alternative to backpropagation enabled by the all-or-none and asynchronous properties of spikes.

In FRI recovery, event times are first converted to integral measurements and then to linear systems in Fourier coefficients. For periodic FRI signals,

$E^k = \text{concat}(E_1^k, E_2^k, \cdots, E_J^k).$7

the filtered signal retains a chosen Fourier subset, and timing-derived measurements yield systems such as

$E^k = \text{concat}(E_1^k, E_2^k, \cdots, E_J^k).$8

after which annihilating-filter or Prony-style recovery estimates $E^k = \text{concat}(E_1^k, E_2^k, \cdots, E_J^k).$9 (Naaman et al., 2021). A related treatment for C-TEM and IF-TEM shows that after kernel filtering,

$M_1=5$0

and both crossing-time measurements and local IF-TEM interval averages produce full-column-rank linear systems in the Fourier coefficients under sufficient event-density conditions (Kamath et al., 2021).

Direct inference without reconstruction has become a distinct line of work. In IF-TEM-based symbol detection, the observation model

$M_1=5$1

links unknown symbols directly to event times, enabling direct symbol decisions from inter-event intervals (Bernardo, 25 Aug 2025). “Message Passing Based Demodulation of the Time-Encoded Digital Modulation Signal” goes further by converting spike intervals into a sparse linear Gaussian model,

$M_1=5$2

where each row of $M_1=5$3 has at most two adjacent nonzero columns for rectangular pulse shaping, yielding a chain-like factor graph. The paper then proposes a Sliding Message Passing algorithm that processes spikes one by one, achieves latency bounded by approximately one spike interval, and has $M_1=5$4 complexity instead of the $M_1=5$5 complexity of pseudo-inverse-based demodulation (Xu et al., 7 Dec 2025).

These examples show that temporal encoding does not imply a single decoding paradigm. Depending on the task, the encoded representation may be consumed by a neural predictor, inverted into a signal model, used to solve linear constraints, or processed directly in the event domain without reconstructing the original waveform.

5. Quantization, hardware realization, and system-level tradeoffs

A large TEM literature concerns what happens after event times are produced: they must be measured, quantized, transmitted, or reconstructed in real hardware. Here, interval statistics and nonideal circuit effects become central.

Compressed IF-TEM work exploits the fact that inter-spike intervals are bounded and, in the cited formulation, sufficiently stationary in time that the known dynamic range

$M_1=5$6

can be subdivided into tighter windows before quantization (Tarnopolsky et al., 2022). In CCIF-TEM and DCIF-TEM, only the within-window residual is always quantized, while the window index is transmitted only when it changes. The paper reports that using the same number of samples and up to $M_1=5$7 additional bits relative to conventional IF-TEM improves reconstruction by $M_1=5$8–$M_1=5$9 dB in the reported error measure, and that for a fixed reconstruction target CIF-TEM can use $M_2=10$0–$M_2=10$1 fewer bits (Tarnopolsky et al., 2022).

A related but more explicitly distribution-aware approach derives the interval density for a class of bandlimited signals: $M_2=10$2 This shows that TEM firing intervals are inherently non-uniformly distributed, which motivates non-uniform quantization rather than uniform quantization (Yashaswini et al., 4 Nov 2025). The paper proposes a power-law companding scheme based on

$M_2=10$3

and reports that to reach $M_2=10$4 dB NMSE, TEM-NUQ needs $M_2=10$5 bits, compared with $M_2=10$6 bits for TEM-UQ, while NUS-NUQ and NUS-UQ require $M_2=10$7 and $M_2=10$8 bits, respectively (Yashaswini et al., 4 Nov 2025).

Hardware-aware TEM design extends beyond quantization. A hardware prototype of a time-encoding sub-Nyquist ADC implements an IF-TEM with bias stage, op-amp integrator, comparator, differentiator, and FET-based reset, and uses the timing relation

$M_2=10$9

together with a robust partial-sum reconstruction model to recover FRI parameters (Naaman et al., 2023). The reported hardware result is recovery error up to $(12\times2, 9\times3)$0 dB while operating at rates approximately $(12\times2, 9\times3)$1 times lower than the Nyquist rate (Naaman et al., 2023).

Recent work pushes further toward practical mixed-signal front ends. The Adaptive Compressed IF-TEM combines adaptive biasing and pre-quantization compression, using a predictor-based local bias

$(12\times2, 9\times3)$2

and an adaptive clockless two-step pulse-shrinking TDC that integrates compression within the TDC itself (Karp et al., 4 Nov 2025). The paper reports at least 3-bit savings out of 9 bits over AIF-TEM and $(12\times2, 9\times3)$3 compression over IF-TEM for fixed recovery MSE with real audio signals (Karp et al., 4 Nov 2025). The Self-Calibrating IF-TEM goes after mismatch rather than bit rate, introducing a practical model with unknown scaling $(12\times2, 9\times3)$4 and discharge time $(12\times2, 9\times3)$5,

$(12\times2, 9\times3)$6

and a two-equation calibration scheme that estimates those parameters during sampling (Mekel et al., 13 Sep 2025). Under its simulation setup, S-IF-TEM reaches average NMSE $(12\times2, 9\times3)$7 dB, close to ideal and genie-aided variants, and the paper reports improvement exceeding $(12\times2, 9\times3)$8 dB over blind reconstruction (Mekel et al., 13 Sep 2025).

A separate design axis is bias shaping. The Linear-Bias IF-TEM introduces a piecewise linear bias

$(12\times2, 9\times3)$9

for bandlimited signals with derivative bound $(9\times3, 7\times5, 6\times7, 1\times1)$0, and gives explicit formulas

$(9\times3, 7\times5, 6\times7, 1\times1)$1

that enforce

$(9\times3, 7\times5, 6\times7, 1\times1)$2

In the reported experiments, conventional IF-TEM, VB-IF-TEM, and LB-IF-TEM are matched to $(9\times3, 7\times5, 6\times7, 1\times1)$3 dB unquantized reconstruction NMSE, requiring $(9\times3, 7\times5, 6\times7, 1\times1)$4, $(9\times3, 7\times5, 6\times7, 1\times1)$5, and $(9\times3, 7\times5, 6\times7, 1\times1)$6 firings respectively, with LB-IF-TEM attaining comparable accuracy with significantly fewer firings (Arora et al., 12 Nov 2025).

6. Neuromorphic and neuroscientific extensions

Temporal encoding also appears as a computational principle in neuromorphic and biological sequence models, where time is not merely a measurement outcome but the primary state variable of computation.

Time-to-Event Margin Propagation (TEMP) is exemplary. It proposes a fully asynchronous AER architecture in which delay, triggering, and sorting inherent in the interconnect itself perform computation, and a TEMP neuron emits an output time $(9\times3, 7\times5, 6\times7, 1\times1)$7 satisfying

$(9\times3, 7\times5, 6\times7, 1\times1)$8

In the differential formulation,

$(9\times3, 7\times5, 6\times7, 1\times1)$9

with analogous expression for $$223 = 12\cdot4 + 9\cdot3 + 9\cdot8 + 7\cdot6 + 6\cdot4 + 1\cdot10.$$0, so synaptic parameters are implemented as delays rather than multiply-and-accumulate weights (R et al., 2023). The paper argues that the resulting time-based encoding yields a spatio-temporal representation capable of encoding many discriminatory patterns, and reports greater than $$223 = 12\cdot4 + 9\cdot3 + 9\cdot8 + 7\cdot6 + 6\cdot4 + 1\cdot10.$$1 accuracy on MNIST in a proof-of-concept TEMP-based CNN (R et al., 2023).

A neuroscientific extension of spiking Temporal Memory makes timing itself a learned representation rather than a fixed consequence of intrinsic time constants. The proposed mechanism represents the duration of sequence elements by sequential activation of element-specific neuronal populations and uses oscillatory background inputs as a clock signal for flexible replay-speed control (Lober et al., 21 May 2026). The paper states that elapsed time is encoded by unique and sparse spatiotemporal patterns of neural activity, suggesting a biologically plausible temporal encoding mechanism for sequence timing and replay-speed modulation (Lober et al., 21 May 2026).

Across these neuromorphic settings, a plausible implication is that a TEM need not be confined to the input layer. TEMP effectively turns every layer into a temporal encoder by remapping event times through learned delays and causal threshold crossings, while the extended sTM model encodes duration through progression across sparse neural states. This broadens TEM from a front-end sensing module to a general principle of temporal representation and computation.

The literature therefore supports a broad but technically coherent view of Temporal Encoding Module. In deep learning, it is a sequence-to-feature front end specialized for temporal context aggregation. In signal processing and spiking systems, it is a Time Encoding Machine that converts analog dynamics into event times. In neuromorphic computation, it becomes a delay-based substrate in which timing is itself the computational currency. The common denominator is explicit temporal representation: whether as embeddings, spike times, inter-event intervals, or sparse spatiotemporal states, the module’s function is to transform temporal structure into a form that downstream mechanisms can exploit (Lebailly et al., 2020, Adam, 2021, R et al., 2023).