Recency Encoding in Temporal Models

• Recency encoding is a family of techniques that operationalize temporal nearness by representing elapsed intervals, freshness, or training order in model inputs.
• Techniques include learnable Fourier embeddings, gated attention mechanisms, and recency-aware sampling, applied in knowledge tracing, translation, retrieval, and recommendation.
• Empirical studies show that models incorporating recency encoding often outperform traditional positional encodings, improving prediction accuracy and handling temporal dynamics.

Searching arXiv for papers on recency encoding and closely related temporal/positional mechanisms. Recency encoding denotes a class of representational, algorithmic, and formal devices that make temporal nearness operational inside a system. In the literature, the term does not refer to a single standardized mechanism. It may denote an explicit embedding of elapsed steps since a concept was last practiced in knowledge tracing, a gated injection of the immediately preceding sentence in neural machine translation, a relative freshness vector in temporal retrieval, a recency-biased supervision distribution in recommendation, an emergent positional attention prior in long-context LLMs, a linearly decodable training-order signal in hidden activations, or a bound restricting updates to the most recent elements in a database-driven transition system (Badran et al., 23 Aug 2025, Kuang et al., 2018, Cao et al., 1 Sep 2025, Petrov et al., 2022, Peysakhovich et al., 2023, Krasheninnikov et al., 17 Sep 2025, Abdulla et al., 2016).

1. Domain-specific meanings of recency encoding

A recurring source of confusion is that “recency” is operationalized differently across subfields. Some uses are explicitly representational, while others are procedural or even semantic.

Setting	Carrier of recency	Operational meaning
Knowledge tracing	KC-specific elapsed distance	Steps since the same KC last appeared
NMT / retrieval	Previous sentence or freshness gap	Neighboring context or document staleness
Recommendation / QA / verification	Sampling bias, question label, or recency bound	Supervision prior, metadata, or admissible updates
Long-context LMs / SSMs	Attention prior or recurrent decay	Internal preference for nearby tokens

In knowledge tracing, recency is the number of original interaction steps since the most recent occurrence of the same KC, computed over the original question-level sequence rather than the KC-expanded sequence (Badran et al., 23 Aug 2025). In neural machine translation, recency is the immediately preceding source sentence and is fused through an inter-sentence gate (Kuang et al., 2018). In temporal retrieval, recency is document freshness measured from a reference time via a relative gap based on the publication or update timestamp $t_d$ (Cao et al., 1 Sep 2025). In recommendation, the method called RSS does not introduce a recency embedding at all; it biases target sampling probabilities toward recent interactions (Petrov et al., 2022). In long-context LMs, recency can appear as an emergent attention prior favoring later documents (Peysakhovich et al., 2023). In controlled fine-tuning studies, recency is a latent activation-space signal encoding when information was acquired during training (Krasheninnikov et al., 17 Sep 2025). In formal verification, recency is an age order over database elements, and only the most recent $k$ elements may be modified in a $k$-recency-bounded run (Abdulla et al., 2016).

This variation makes one general point precise: recency encoding is not synonymous with positional encoding. It may encode elapsed reuse, local discourse continuity, freshness, training chronology, or admissible update scope, depending on the modeling objective.

2. Knowledge tracing: KC-specific elapsed-distance embeddings

In knowledge tracing, Recency Encoding was introduced as an embedding-level modification for models operating on expanded KC sequences. The encoded variable is “the number of time steps since the most recent occurrence of the same KC within the original student interaction sequence,” with first occurrence handled by the sentinel value $d=0$. The distinction between original interaction sequence and expanded KC sequence is central: if a KC appeared in question $t-7$ and reappears in question $t$, the recency distance is $7$ regardless of how many KCs were attached to those questions after expansion (Badran et al., 23 Aug 2025).

The representation is implemented through learnable Fourier features. For a scalar distance $d \in \mathbb{R}$,

$$\gamma(d) = \left[ \cos(d \mathbf{w}_f + \mathbf{b}_f),\ \sin(d \mathbf{w}_f + \mathbf{b}_f) \right],$$

where $\mathbf{w}_f \sim \mathcal{N}(0,1)$ and $k$0. The projected embedding is

$k$1

with $k$2 described as a linear layer followed by GeLU. The paper does not typeset a standalone symbolic definition of $k$3, but a reconstructed form is

$k$4

This reconstructed formula is inferred from the text rather than printed as such.

The method is explicitly contrasted with positional encoding. A learned positional embedding such as SAKT’s $k$5 maps an absolute timestep $k$6 to a vector, whereas Recency Encoding maps a KC-specific elapsed distance $k$7 to a vector. The paper’s summary is concise: positional encoding answers “where am I in the sequence?”; recency encoding answers “how long since this concept was last practiced?” That distinction is motivated by forgetting and repetition effects, which are more directly tied to concept recurrence spacing than to absolute position.

The mechanism is architecture-agnostic because it is applied at the embedding level. In DKT it is incorporated into input embeddings before the recurrent layer. In SAKT it replaces the original positional encoding. In AKT it is added exclusively to the value vectors while preserving AKT’s original monotonic attention. The same paper pairs this with a leakage-free masking strategy using a dedicated MASK label for all but the last KC of a question in the expanded input sequence, so recency and leakage prevention address different failure modes: the former improves temporal representation, while the latter prevents label leakage.

Empirically, the recency-enhanced variants improve or mostly improve over the leakage-controlled baselines on ASSISTments2009, Algebra2005, Riiid2020, and Duolingo2018. The clearest result is AKT-ML$k$8: $k$9 versus AKT-ML: $k$0. For SAKT, SAKT-ML$k$1: $k$2 versus SAKT-ML: $k$3, which the paper explicitly interprets as evidence that recency encoding performed better than positional encoding. For DKT, DKT-ML$k$4: $k$5 versus DKT-ML: $k$6, with a slight drop on Riiid20 but gains elsewhere, especially on Duolingo2018. The paper also compares learnable Fourier recency to fixed Fourier features and concludes that fixed features can help, but learnable recency encoding is generally stronger and more consistent.

3. Architectural formulations in sequence modeling and retrieval

A distinct use of recency encoding appears in document-level neural machine translation. The inter-sentence gate model takes the current source sentence $k$7 together with the immediately preceding source sentence $k$8, encodes both with the same encoder, computes attention contexts $k$9 and $d=0$0, and injects them into decoder-state updates through an elementwise gate

$d=0$1

The decoder state is then updated as

$d=0$2

Only the current-sentence context $d=0$3 is used directly for word prediction, so the previous sentence shapes translation indirectly through the hidden state. This defines recency narrowly as one-step source-side sentence history. On NIST Chinese-English translation, the model reaches average BLEU $d=0$4 versus $d=0$5 for RNNSearch, with ablations showing $d=0$6 for +NULL, $d=0$7 for $d=0$8, and $d=0$9 for +RV (Kuang et al., 2018).

A more explicitly temporal formulation appears in SIREN-RoPE, which replaces fixed ordinal RoPE angles with a fused temporal-plus-ordinal angle

$t-7$0

The timestamp features are

$t-7$1

with $t-7$2 and $t-7$3. Here recency is only one component of a broader temporal geometry: the model retains a learnable amount of ordinal recency through $t-7$4 while also encoding continuous timestamps, cyclical daily and weekly structure, and long-range temporal trends. The learned gate $t-7$5 drops from $t-7$6 to about $t-7$7, which the paper interprets as the model relying mostly on temporal modulation rather than discrete ordinal recency. Reported gains include Contribution NE $t-7$8, Like AUC $t-7$9, and LongDwell AUC $t$0 (Cheng et al., 27 Apr 2026).

Temporal retrieval supplies another explicit encoding pattern. Re3 defines a query and document as

$t$1

and separates temporal alignment from freshness. Recency is based on the document publication or update timestamp via

$t$2

measured in days. It is encoded with multi-frequency sinusoidal features

$t$3

with $t$4. The time-aware document representation is then fused with the semantic representation through a learnable soft gate: $t$5 On Re$t$6-Rec, Re$t$7 reaches $t$8; on Re$t$9-Hyb it reaches $7$0, $7$1, and $7$2. The ablation without the learnable gate drops Re$7$3-Rec $7$4 from $7$5 to $7$6 and Re$7$7-Hyb $7$8 from $7$9 to $d \in \mathbb{R}$0, indicating that relative-gap recency features are useful only when their contribution is balanced against semantic matching (Cao et al., 1 Sep 2025).

4. Recency as supervision, metadata, and formal constraint

Not all recency-aware methods encode recency in latent vectors. In sequential recommendation, RSS is explicitly described as not introducing “an explicit recency encoding vector or embedding.” Instead, it injects recency through the sampling distribution over target positions: $d \in \mathbb{R}$1 with the main experimental choice

$d \in \mathbb{R}$2

and target count

$d \in \mathbb{R}$3

This is a recency-aware sampling strategy rather than a representation-level encoding. The method is applied to GRU4Rec, Caser, and SASRec, and on MovieLens-20M the paper reports, for SASRec, continuation + BCE $d \in \mathbb{R}$4, $d \in \mathbb{R}$5 versus RSS + BCE $d \in \mathbb{R}$6, $d \in \mathbb{R}$7; it also states that SASRec-RSS-$d \in \mathbb{R}$8Rank reaches $d \in \mathbb{R}$9 in $$\gamma(d) = \left[ \cos(d \mathbf{w}_f + \mathbf{b}_f),\ \sin(d \mathbf{w}_f + \mathbf{b}_f) \right],$$0 hour, compared with BERT4Rec-$$\gamma(d) = \left[ \cos(d \mathbf{w}_f + \mathbf{b}_f),\ \sin(d \mathbf{w}_f + \mathbf{b}_f) \right],$$1h at $$\gamma(d) = \left[ \cos(d \mathbf{w}_f + \mathbf{b}_f),\ \sin(d \mathbf{w}_f + \mathbf{b}_f) \right],$$2 (Petrov et al., 2022).

In question answering, recency can be modeled as metadata attached to the question rather than to the sequence state. RecencyQA introduces a recency–stationarity taxonomy in which recency is the expected temporal stability of an answer and stationarity is the stability of that recency behavior over time. The recency label space has 12 classes: An-Hour, A-Few-Hours, A-Day, A-Few-Days, A-Week, A-Few-Weeks, A-Month, A-Few-Months, A-Year, A-Few-Years, Many-Years, and Never. The dataset contains $$\gamma(d) = \left[ \cos(d \mathbf{w}_f + \mathbf{b}_f),\ \sin(d \mathbf{w}_f + \mathbf{b}_f) \right],$$3 questions, $$\gamma(d) = \left[ \cos(d \mathbf{w}_f + \mathbf{b}_f),\ \sin(d \mathbf{w}_f + \mathbf{b}_f) \right],$$4 stationary questions, and $$\gamma(d) = \left[ \cos(d \mathbf{w}_f + \mathbf{b}_f),\ \sin(d \mathbf{w}_f + \mathbf{b}_f) \right],$$5 non-stationary questions. Human evaluation reports $$\gamma(d) = \left[ \cos(d \mathbf{w}_f + \mathbf{b}_f),\ \sin(d \mathbf{w}_f + \mathbf{b}_f) \right],$$6 strict accuracy for recency labeling, $$\gamma(d) = \left[ \cos(d \mathbf{w}_f + \mathbf{b}_f),\ \sin(d \mathbf{w}_f + \mathbf{b}_f) \right],$$7 tolerant accuracy, and $$\gamma(d) = \left[ \cos(d \mathbf{w}_f + \mathbf{b}_f),\ \sin(d \mathbf{w}_f + \mathbf{b}_f) \right],$$8 stationarity accuracy. The dynamic transition experiment on non-stationary questions yields transition accuracy $$\gamma(d) = \left[ \cos(d \mathbf{w}_f + \mathbf{b}_f),\ \sin(d \mathbf{w}_f + \mathbf{b}_f) \right],$$9 in zero-shot, $\mathbf{w}_f \sim \mathcal{N}(0,1)$0 in few-shot, and $\mathbf{w}_f \sim \mathcal{N}(0,1)$1 in chain-of-thought for Gemma 3 (27B), showing that context-conditioned recency is difficult for current LLMs (Piryani et al., 17 Mar 2026).

A formally different use occurs in database-driven verification. A $\mathbf{w}_f \sim \mathcal{N}(0,1)$2-recency-bounded run permits actions to inspect the whole current database but allows only the most recent $\mathbf{w}_f \sim \mathcal{N}(0,1)$3 active-domain elements to be modified. With an injective sequence numbering $\mathbf{w}_f \sim \mathcal{N}(0,1)$4, the paper defines $\mathbf{w}_f \sim \mathcal{N}(0,1)$5 as the maximal set of at most $\mathbf{w}_f \sim \mathcal{N}(0,1)$6 active elements whose sequence numbers are larger than those of all active elements outside the set. Under this restriction, more runs can be verified by increasing the recency bound, and recency-bounded model checking of DMS against MSO-FO becomes decidable by reduction to satisfiability of MSO over nested words (Abdulla et al., 2016).

These cases show that recency encoding can live in three different places: the sampling distribution, the problem specification, or the transition semantics. The encoded object is still temporal nearness, but not necessarily as a learned vector.

5. Emergent and decodable recency in model internals

In long-context LLMs, recency may emerge as a positional attention prior rather than being hand-designed as an input feature. One study shows that during first-token decoding, attention shifts toward recent documents as context length grows, even though relevant documents still receive more attention than distractors at the same position. This motivates “attention sorting”: perform one decoding step, aggregate attention per document, sort documents by attention with highest attention going last, optionally repeat, and then generate the answer with the reordered context. The method improves accuracy on SynthWiki for all three studied open-source models; for WizardCoder and YaRN it more than doubles accuracy at $\mathbf{w}_f \sim \mathcal{N}(0,1)$7 context length, while TogetherLlama recovers most degradation in about $\mathbf{w}_f \sim \mathcal{N}(0,1)$8 iterations and the other models continue improving out to $\mathbf{w}_f \sim \mathcal{N}(0,1)$9 attention sorts (Peysakhovich et al., 2023).

A different internal notion of recency appears in activation-space analyses of training chronology. Sequential fine-tuning of Llama-3.2-1B on six disjoint datasets $k$00 yields dataset centroids whose projections are arranged exactly in the order of training and lie on a straight line in a 2D subspace. Linear probes trained on single activation vectors achieve over $k$01 accuracy for $k$02 vs. $k$03, and an additional supervised stage-reporting task reaches $k$04 accuracy versus a $k$05 random baseline. The signal is robust across prompts, runs, model families, and full fine-tuning versus LoRA, and the paper argues that it is not reducible to simple activation magnitudes, losses, or model confidence. After $k$06 epochs of subsequent mixed-data training, $k$07 vs. $k$08 probe accuracy decays only from $k$09 to about $k$10 in the synthetic setting and from about $k$11 to about $k$12 in the natural setting, so the recency trace persists even when all examples have become equally recent under later mixed training (Krasheninnikov et al., 17 Sep 2025).

These studies are notable because recency is neither an explicit feature channel nor a training objective. It is either a bias in where attention naturally flows or a linearly recoverable latent variable in hidden-state geometry. This suggests that recency can be an emergent property of model internals even when it is absent from the task specification.

6. Recency bias, memory dynamics, and limitations

State-space models make recency especially visible because their recurrent form directly exposes lag-dependent decay. In Mamba, the per-dimension update

$k$13

unrolls into

$k$14

The paper interprets this product as an effective memory kernel: older inputs undergo more decay steps, so recent inputs dominate. On structured recall tasks, Mamba shows a U-shaped serial-position curve. The recency advantage is fragile, however. Adding distractor tokens “disrupts the accuracy of the recall, especially in the recent positions,” and the paper concludes that recency is a short-term-memory phenomenon sustained by state transitions that saturate over time. The learned discretization parameter $k$15 modulates this effect: smaller values support slower decay and longer memory retention, while larger values correspond to faster forgetting and stronger immediate input influence (Airlangga et al., 18 Jun 2025).

A complementary SSM analysis gives a theorem-level statement of recency bias. For diagonal $k$16, the paper proves

$k$17

and in the appendix states the stronger bound

$k$18

This formalizes recency as exponential attenuation of token influence with distance. The same work argues that depth partially compensates by expanding effective receptive field, but deeper SSMs inevitably over-smooth token representations. Its proposed mitigation is to polarize two transition channels,

$k$19

thereby preserving one perfectly persistent channel and one purely local channel. On associative recall, the 4-layer polarized model reaches $k$20, $k$21, and $k$22 accuracy for $k$23 key-value pairs, versus $k$24, $k$25, and $k$26 for the 4-layer default Mamba (Wang et al., 2024).

Yet recency does not exhaust the serial-position behavior of SSMs. In a controlled study of S4 on a binary memory verification task, the strongest empirical effect is primacy rather than simple monotone recency. Accuracy is highest for study items presented at the beginning of the sequence, and “the S4's accuracy for the most recently presented study items was lowest when they were queried immediately after their initial presentation.” The paper identifies learned concentration of small $k$27 values, roughly $k$28, as a key factor associated with long-memory behavior, but it explicitly states that this is only a necessary condition and does not fully explain why late items are remembered less accurately despite the exponentially decaying measure (Morita, 19 Feb 2025).

Taken together, these results delimit the scope of recency encoding. It is not uniformly beneficial, not reducible to absolute position, and not always equivalent to wall-clock time. In KT it is original interaction-step distance since the same KC; in retrieval it is a day-gap from $k$29; in recommendation it can be only a sampling prior; in verification it is an age-bounded update regime; and in long-context LMs or SSMs it can become a bias that suppresses distant evidence. A plausible synthesis is that recency encoding is best understood not as one mechanism but as a family of temporal inductive biases, each useful only insofar as its notion of “recent” matches the task’s operative time scale and failure modes.