---
title: Leakage-Free State Prediction
url: https://www.emergentmind.com/topics/leakage-free-state-prediction
type: topic
---

# Leakage-Free State Prediction

Searching arXiv for papers on leakage-free state prediction and closely related formulations.
arxiv_search(query="leakage-free state prediction OR state leakage prediction", max_results=10)
Leakage-free state prediction, as suggested by several otherwise distinct research literatures, denotes prediction, transmission, or control schemes in which the inferred or communicated state is not contaminated by information that lies outside the intended causal, temporal, architectural, or physical interface. In the cited work, leakage may mean \(L_e(c)=\frac1n I(S^n;Y^n)\) in a state-dependent channel, a suppressed image sideband in continuous-variable quantum key distribution, transitions out of a target quantum state, label leakage in knowledge tracing, temporal leakage in cascade evaluation, private information exposed in chain-of-thought traces, or secret data transiently propagated through microarchitectural predictors [1812.07026] [2205.07245] [2112.00203] [2508.17092] [2510.25348] [2511.07772] [1905.09100].

## 1. Conceptual scope

Across these literatures, “state” has several technical meanings: a random channel state \(S\), a coherent-state displacement \(\alpha_k\), the amplitude of a target quantum state \(P(t)\), a student’s latent knowledge trajectory inferred from interaction history, the evolving state of an information cascade, the predicted-risk/outcome law \(\mathrm{Law}((p,y))\), or the hidden architectural and transformer states that mediate execution and reasoning. The unifying constraint is that a predictor, controller, or communication system should neither exploit illegitimate information nor expose state information through unintended channels [1812.07026] [2205.07245] [2112.00203] [2508.17092] [2510.25348] [2606.11267] [1905.09100] [2511.07772].

| Domain | State notion | Leakage mechanism or control |
|---|---|---|
| State-dependent channels | \(S^n\) | \(L_e(c)=\frac1n I(S^n;Y^n)\), empirical coordination |
| CV-QKD | \(|\alpha_k\rangle\) | Baseband IQ modulation removes the image tone |
| Quantum control | \(P(t)=\langle A(t)|X(t)\rangle\) | Suppress the kernel \(g'(t,s)\) with \(R_L(t)\) |
| Knowledge tracing | \((q_t,r_t)\), \((c_i,r_i)\) | MASK label prevents intra-question label leakage |
| Cascade prediction | \(\Delta P_c\) | Time-ordered splitting prevents future information leakage |
| Output-only auditing | \(\mathrm{Law}((p,y))\) | Unit-purity head, ENB, AUC ceiling |
| Systems and LLMs | register taint, hidden activations | dummy forwarding or steering vectors |

A further commonality is methodological. Some formulations make leakage a first-class objective in the optimization or achievability region, as in rate–equivocation–coordination theory. Others remove a physical side-channel, redesign the evaluation protocol to respect chronology, or intervene directly in hidden state. This suggests that leakage-free state prediction is best treated not as a single theory, but as a family of constraints on what information may be used, inferred, or revealed.

## 2. Information-theoretic formulations: masking, coordination, and state inference

In state-dependent communication, the basic setup is a memoryless channel with input \(X\in\mathcal X\), output \(Y\in\mathcal Y\), and random state \(S\in\mathcal S\) drawn i.i.d. \(\sim P_S\). Under causal state knowledge, the encoder symbol satisfies \(X_i=f_i(M,S^i)\); under strictly-causal state knowledge, \(X_i=f_i(M,S^{i-1})\). The decoder observes \(Y^n\), produces a message estimate \(\hat M\), and may generate an action sequence \(V^n\in\mathcal V^n\) so that the joint empirical frequency of \((S^n,X^n,Y^n,V^n)\) is forced close to a target \(Q_{SXYV}\) [1812.07026].

The principal leakage metric is
\[
L_e(c)=\frac1n I(S^n;Y^n).
\]
A leakage level \(E\) is achievable if \(\lvert L_e(c)-E\rvert\le \epsilon\). The same framework incorporates empirical coordination and the “core of the receiver’s knowledge,” captured by \((Y_i,W_{1,i},W_{2,i})\), where \(W_1\) and \(W_2\) are auxiliary variables. Mutual-information terms involving \((W_1,W_2,Y)\) then quantify what the decoder can infer about \((S^n,X^n,V^n)\) [1812.07026].

For causal encoding, Theorem II.3 states that a triple \((R,E,Q_{SXYV})\) is achievable if and only if there exist auxiliaries \((W_1,W_2)\) and a joint law
\[
P_S\,Q_{W_1}\,Q_{W_2|S,W_1}\,Q_{X|S,W_1}\,P_{Y|X,S}\,Q_{V|Y,W_1,W_2}
\]
such that
\[
I(S;W_1,W_2,Y)\le E\le H(S),
\qquad
R+E\le I(W_1,S;Y),
\]
with cardinality bounds \(\lvert W_1\rvert,\lvert W_2\rvert\le|\mathcal S\times\mathcal X\times\mathcal V|+1\) [1812.07026]. In this formulation, leakage-free prediction is not absolute invisibility; it is the controlled selection of an achievable leakage level consistent with reliability and empirical coordination.

The coding construction uses a Block-Markov structure. The encoder quantizes the previous block’s state sequence into a bin index \(L_{b-1}\), chooses an index \(K_b\) so that \((S^n_{b-1},W_1,W_2)\) are jointly typical, and sends codeword \(X^n_b\) drawn i.i.d. from \(Q_{X|SW_1}\). The decoder recovers \((M_b,L_{b-1},K_b)\) by joint-typicality, reconstructs \(V^n_b\sim Q_{V|YW_1W_2}\), and learns the bin of \(S^n_{b-1}\). Balancing the message, binning, and coordination rates yields \(\frac1n I(S^n;Y^n)\approx E\) while achieving \((R,Q)\) [1812.07026].

The framework extends to two-sided state information and noisy feedback. In the former, one replaces the leakage term by \(I(U,S;W_1,W_2;Y,Z)\le E\) and the sum constraint by \(R+E\le I(W_1,U,S;Y,Z)\). In the latter, the rate becomes
\[
R\le I(W_1,W_2;Y_1)-I(W_2;S,Y_2\mid W_1),
\]
while the same leakage bound \(I(S;W_1,W_2,Y_1)\le E\le H(S)\) and overall \(R+E\le I(W_1,S;Y_1)\) remain. The paper also formulates a zero-sum channel-state estimation game in which the encoder seeks to maximize the decoder’s distortion, with Sion’s theorem yielding a saddle point and a single distortion–rate function \(D^*(R)\) [1812.07026].

## 3. Physical side-channel removal in continuous-variable quantum key distribution

In continuous-variable QKD based on coherent states, a state-preparation side-channel was identified in the form of information leakage about the transmitted quantum state during modulation. The modulation leakage-free architecture of [2205.07245] removes this vulnerability by abandoning RF up-conversion and using a baseband modulation approach with an in-phase and quadrature modulator for state preparation, radio frequency heterodyne detection, and carefully designed digital signal processing for state measurement.

Alice begins with two real classical waveforms \(I(t)\) and \(Q(t)\), each carrying independent Gaussian random variables drawn from \(\mathcal N(0,V_A)\). Instead of up-converting \(I,Q\) to an RF frequency \(\omega\) and generating an optical single-sideband at \(\Omega\pm\omega\), Alice drives a dual-nested Mach–Zehnder IQ modulator directly with baseband voltages
\[
V_{\mathrm{RF1}}(t)\propto I(t),\qquad V_{\mathrm{RF2}}(t)\propto Q(t),
\]
so that no RF up-conversion is performed and the optical carrier remains at the laser frequency \(\Omega\). In the small-signal limit, allowing for DC-bias errors \(\phi_1,\phi_2\) and finite carrier suppression \(\Delta\), the modulator output is
\[
E_{\mathrm{out}}(t)\simeq \tfrac12[\mu I(t)+j\mu Q(t)+\phi_1+j\phi_2]e^{j\Omega t}.
\]
Crucially, there is no second “image” tone at \(\Omega\pm\omega\); the entire classical waveform \(\alpha(t)=I(t)+jQ(t)\) rides at the single optical frequency \(\Omega\) [2205.07245].

After attenuation to the quantum level, each symbol interval \(T\) yields an approximate coherent state \(|\alpha_k\rangle\) with
\[
\alpha_k=\mu[I_k+jQ_k]/2,\qquad I_k,Q_k\sim\mathcal N(0,V_A).
\]
In phase space,
\[
\langle \Delta X^2\rangle=\langle \Delta P^2\rangle=V_A+1,
\]
where \(1\) is the shot-noise unit. The prepared ensemble is
\[
\rho_A=\int d^2\alpha\,P(\alpha)\,|\alpha\rangle\langle \alpha|,
\qquad
P(\alpha)=\frac{1}{\pi V_A}\exp[-|\alpha|^2/V_A],
\]
with covariance matrix
\[
\Gamma_A=
\begin{pmatrix}
V_A+1 & 0\\
0 & V_A+1
\end{pmatrix}.
\]
For a Gaussian channel of transmittance \(\eta\) and excess noise \(\xi\), Bob’s variance is
\[
V_B=\eta V_A+1+\eta \xi,
\]
and the joint covariance matrix \(\Gamma_{AB}\) has diagonal block \(V_A+1\) and correlation block \(\sqrt{\eta}\,V_A\), repeated for \(P\) [2205.07245].

The security proof works in the asymptotic limit with reverse reconciliation. The key rate per use is
\[
K=\beta I_{AB}-\chi_{BE},
\qquad
I_{AB}=\log_2\!\left[\frac{V_B}{1+\eta\xi}\right].
\]
For a Gaussian attack,
\[
\chi_{BE}=S(E)-S(E|B)=G(\nu_+)+G(\nu_-)-G(\nu_B),
\]
where
\[
\nu_{\pm}=\tfrac12\Big[\sqrt{(V_A+1+\eta\xi)^2-4\eta V_A}\pm(V_A+1-\eta\xi)\Big],
\qquad
\nu_B=V_B-1,
\]
and
\[
G(x)=\frac{x+1}{2}\log_2\frac{x+1}{2}-\frac{x-1}{2}\log_2\frac{x-1}{2}.
\]
Thus,
\[
K=\beta\log_2\!\left[\frac{\eta V_A+1+\eta\xi}{1+\eta\xi}\right]-\chi_{BE}.
\]
Because no image sideband is ever created, Eve cannot steal any extra tone, and the ideal security proof of Gaussian-modulated CV-QKD is restored with no extra side-channel terms [2205.07245].

The receiver DSP performs whitening of electronic plus vacuum noise spectra; frequency-offset recovery via a strong pilot tone at \(\sim 60\,\mathrm{MHz}\); carrier-phase tracking using an unscented Kalman filter; high-pass filtering at \(190\,\mathrm{kHz}\) with a 5th-order Butterworth response; and root-raised-cosine matched filtering with roll-off \(0.2\), followed by down-sampling to \(20\,\mathrm{MBaud}\). The implementation used a CW \(1550\,\mathrm{nm}\) laser with \(\lesssim 100\,\mathrm{Hz}\) linewidth, \(V_A=0.27\,\mathrm{PNU}\), AWG and ADC at \(1\,\mathrm{GS/s}\), a \(20\,\mathrm{km}\) SMF channel with physical loss \(\simeq 4\,\mathrm{dB}\), experimentally inferred \(\eta\approx 0.24\), excess noise \(\xi\approx 0.00072\), and shot-noise clearance \(\simeq 15\,\mathrm{dB}\). With an 8-dimensional MET-LDPC code of base code rate \(0.02\), punctured for \(\beta\approx 93\%\) at \(\mathrm{SNR}\approx 0.044\), the frame-error rate was \(\simeq 0.22\). For finite-size composable security, \(N=10^9\) states yielded a secret-key fraction \(\simeq 0.007\,\mathrm{bit/symbol}\) over \(20\,\mathrm{km}\) [2205.07245].

## 4. Leakage-free paths in quantum dynamics and control

A distinct quantum use of the concept appears in the derivation of an exact one-component equation of motion for the probability amplitude of a chosen target time-dependent state. Starting from the general linear equation
\[
\frac{dX}{dt}=M(t)X(t),
\]
one selects a one-dimensional \(P\)-subspace spanned by a normalized target state \(|A(t)\rangle\), defines \(P(t)=\langle A(t)|X(t)\rangle\), and lets \(Q(t)\) denote the complementary components. Writing
\[
X=[P;Q],\qquad M=\begin{bmatrix}h&R\\ W&D\end{bmatrix},
\]
yields
\[
\frac{dP}{dt}=h(t)P(t)+R(t)Q(t),
\qquad
\frac{dQ}{dt}=W(t)P(t)+D(t)Q(t).
\]
Integrating out \(Q\) with propagator \(G(t,s)=T\exp[\int_s^t D(\tau)\,d\tau]\) and \(Q(0)=0\) gives
\[
Q(t)=\int_0^t G(t,s)W(s)P(s)\,ds
\]
and hence
\[
\frac{dP}{dt}=h(t)P(t)+\int_0^t [R(t)G(t,s)W(s)]P(s)\,ds.
\]
Defining \(g(t,s)=R(t)G(t,s)W(s)\) and factoring \(P(t)=p(t)\exp[\int_0^t h(\tau)\,d\tau]\), one obtains the one-component equation
\[
\partial_t p(t)=\int_0^t g'(t,s)\,p(s)\,ds,
\qquad
g'(t,s)=\exp\Bigl[\int_s^t h(\tau)\,d\tau\Bigr]R(t)G(t,s)W(s)
\]
[2112.00203].

In this formulation, all leakage out of the target path is encoded in the kernel \(g'(t,s)\). The leakage-elimination operator is introduced by decomposing the Hamiltonian or super-operator into block-diagonal and block-off-diagonal parts, \(H_d=h\oplus D\) and \(L=\bigl[\begin{smallmatrix}0&R\\ W&0\end{smallmatrix}\bigr]\), and adding
\[
R_L(t)=c(t)\bigl[|\phi_0(t)\rangle\langle\phi_0(t)|-\sum_{n>0}|\phi_n(t)\rangle\langle\phi_n(t)|\bigr]
=c(t)\,[2|\phi_0\rangle\langle\phi_0|-I].
\]
Because \(\{R_L,L\}=0\) and \([R_L,H_d]=0\), the added term “parity kicks out” the off-diagonal leakage \(L\) nonperturbatively. Here \(c(t)\) is an arbitrary bounded real-valued control function [2112.00203].

A sufficient condition for keeping the system on the target path \(P(t)\approx 1\) is
\[
\int_0^t g'(t,s)p(s)\,ds\approx 0.
\]
Equivalently, with
\[
C(t)\equiv i\int_0^t h(\tau)\,d\tau,
\]
one seeks the phase factor \(e^{-iC(s)}\) to be sufficiently rapidly oscillating on \(0\le s\le t\) so that, by the Riemann–Lebesgue lemma,
\[
\int_0^t e^{-iC(s)}g(t,s)p(s)\,ds\longrightarrow 0.
\]
This produces the paper’s “universal leakage-free path” condition for both closed and open systems [2112.00203].

The framework unifies several standard control limits. In the \(\delta\)-pulse limit of \(c(t)\), one recovers bang–bang parity kicking. Replacing fast kicks by repeated projective measurements \(P=|A(s)\rangle\langle A(s)|\) yields the quantum Zeno limit. In an adiabatic frame, the kernel acquires rapidly oscillating factors \(\exp[-i\int_s^t(E_0-E_n)\,d\tau]\), and the usual adiabatic condition \(|\langle \dot E_0|E_n\rangle/(E_n-E_0)|\ll 1\) appears as the requirement that the oscillatory integral vanish. The same control term can therefore accelerate adiabatic passage by effectively enlarging the phase accumulation [2112.00203].

Two explicit examples were given. For a two-level system with
\[
H(t)=\tfrac12\Delta(t)\sigma_z+\tfrac12\Omega(t)\sigma_x,
\]
adding \(R_L(t)=c(t)H(t)\) in the lab frame shifts \(h(t)\to h(t)+c(t)E(t)/2\), and choosing \(c(t)\) so that \(\int_0^t c(\tau)E(\tau)/2\,d\tau\) is large and oscillatory suppresses the kernel and enforces accelerated adiabatic following. For a pure-dephasing spin coupled to a bosonic bath, parity kicks \(R_L(t)=c(t)[|0\rangle\langle 0|-|1\rangle\langle 1|]\) multiply the kernel by \(\exp[-i\int_s^t c(\tau)\,d\tau]\), and if this phase oscillates rapidly on the bath correlation time, the qubit remains in \(|0\rangle\) with unity probability [2112.00203].

## 5. Leakage-free predictive modeling in machine learning

In machine learning, leakage-free state prediction is typically a question of respecting temporal causality in evaluation and preventing labels from re-entering the input representation. One line of work treats temporal leakage in information cascade popularity prediction. Another addresses label leakage in Knowledge Tracing, where a student’s future performance is predicted from a sequence of past interactions [2510.25348] [2508.17092].

For information cascades, the central criticism is that random cascade-based splits allow models to access future temporal patterns, yielding unrealistic results. The proposed remedy is a strict chronological partition of the event timeline \([t_{\min},t_{\max}]\) into four equal-length, non-overlapping intervals with boundaries \(t^{(1)},\dots,t^{(5)}\). Training input uses \([t^{(1)},t^{(2)}]\) and training target \((t^{(2)},t^{(3)}]\); validation uses \([t^{(2)},t^{(3)}]\) and \((t^{(3)},t^{(4)}]\); test uses \([t^{(3)},t^{(4)}]\) and \((t^{(4)},t^{(5)}]\). The target is the incremental popularity
\[
\Delta P_c=\lvert G^c(t_0^c+\Delta t_2)\rvert-\lvert G^c(t_0^c+\Delta t_1)\rvert.
\]
CasTemp represents each propagation event \((u_k,t_k)\) as \(\mathbf e_k=[\mathbf h_{u_k};\mathrm{TimeEnc}(t_k)]\), processes self-cascade and cross-cascade temporal walks with a bidirectional GRU, applies attention with time-aware decay \(\alpha_k=\exp(-\lambda(t_{\max}-t_k))\), and augments the resulting representation with a competition graph encoder based on Jaccard edge weights
\[
w_{ij}=\frac{|U_i\cap U_j|}{|U_i\cup U_j|}.
\]
The popularity predictor is an MLP with a Softplus output and MSLE objective [2510.25348].

Under time-ordered splits, CasTemp achieved MSLE \(1.171\) versus the best baseline \(1.206\) on Twitter, \(1.475\) versus \(1.685\) on Weibo, \(1.926\) versus \(2.283\) on APS, and \(0.685\) versus \(3.085\) on Taoke. For Taoke conversion prediction, the results were MSLE \(1.934\) versus \(8.877\), MALE \(0.767\) versus \(2.312\), and Hit@40 \(54.5\%\) versus \(16.7\%\). Per-epoch training time on Twitter was \(\sim 1.2\,\mathrm{s}\) for CasTemp, compared with \(\sim 45\,\mathrm{s}\) for CasFlow and \(\sim 150\,\mathrm{s}\) for CasDo, amounting to up to \(125\times\) speedup [2510.25348].

In Knowledge Tracing, Badran and Preisach describe the task using interactions \((q_t,r_t)\), with \(r_t\in\{0,1\}\), or at the knowledge-concept level \((c_i,r_i)\) after expanding each question through a mapping \(m:Q\to 2^D\). Leakage arises when a question maps to multiple KCs and the true label for one KC becomes visible while predicting another KC from the same question. The proposed remedy reserves a special label \(\mathrm{MASK}\notin\{0,1\}\). If a question expands to several KCs, all earlier KCs receive \(\mathrm{MASK}\) and only the final KC retains the true label:
\[
\tilde r_i=
\begin{cases}
r_i & \text{if } c_i \text{ is the last KC of its question},\\
\mathrm{MASK} & \text{otherwise}.
\end{cases}
\]
The input embedding becomes
\[
x_i=\mathrm{Embed}(c_i)+\mathrm{Embed}(\tilde r_i).
\]
This is complemented by Recency Encoding, where \(d_i=i-\mathrm{last\_index}(c_i)\) is mapped via learnable Fourier features
\[
\gamma(d)=[\cos(d\cdot w_f+b_f),\sin(d\cdot w_f+b_f)]\in\mathbb R^D
\]
and then projected by an MLP into the model embedding space [2508.17092].

The method was integrated into DKT, DKT+, AKT, and SAKT. On ASSIST09 and CorrAS09, the masked variants substantially altered performance relative to leakage-prone baselines: DKT improved from \(0.6990\) to \(0.7185\) on ASSIST09 and from \(0.6312\) to \(0.7163\) on CorrAS09; AKT improved from \(0.7334\) to \(0.7543\) on ASSIST09 and from \(0.6361\) to \(0.7552\) on CorrAS09; SAKT improved from \(0.6946\) to \(0.7166\) on ASSIST09 and from \(0.6336\) to \(0.7189\) on CorrAS09. Adding recency further improved masked variants, including AKT-ML\(^d\) from \(0.7543\) to \(0.7566\) on ASSIST and DKT-ML\(^d\) from \(0.8681\) to \(0.8901\) on Duolingo [2508.17092].

Taken together, these works formalize two distinct but related constraints. Temporal leakage violates the chronology of the prediction task. Label leakage violates the conditional information set of the learner. Leakage-free state prediction in ML therefore depends both on the split protocol and on the embedding or feature-construction pipeline.

## 6. Output-only auditing and the limits of leak detection

A complementary question is whether leakage can be detected from predictions and outcomes alone. In binary prediction, the decision-theoretic framework of [2606.11267] treats any leakage diagnostic as a functional of the joint law
\[
P=\mathrm{Law}((p,y))
\quad\text{on }[0,1]\times\{0,1\},
\]
which, under calibration, factorizes as
\[
P(dp,y=1)=p\,F(dp),\qquad P(dp,y=0)=(1-p)\,F(dp),
\qquad F=\mathrm{Law}(p).
\]
Net benefit at threshold \(\tau\in(0,1)\) is
\[
\mathrm{NB}(\tau)=\frac1n\sum_{i=1}^n\Bigl[y_i\mathbf 1\{p_i\ge \tau\}-(1-y_i)\mathbf 1\{p_i\ge \tau\}\frac{\tau}{1-\tau}\Bigr],
\]
and integrating \(\mathrm{NB}(\tau)\) against a density \(\eta(\tau)\) yields
\[
\mathrm{ENB}_\eta=\int_0^1 \mathrm{NB}(\tau)\eta(\tau)\,d\tau
=\frac1n\sum_{i=1}^n [y_i H(p_i)-(1-y_i)G(p_i)],
\]
where
\[
H(p)=\int_0^p \eta(t)\,dt,\qquad
G(p)=\int_0^p \eta(t)\frac{t}{1-t}\,dt.
\]
The weighting density tunes sensitivity to leakage that appears only in particular risk ranges [2606.11267].

The central impossibility theorem concerns broad-calibrated leakage. If a leaky model is post-hoc recalibrated so that it exactly matches an honest model’s calibration and discrimination, then no statistic on \((p,y)\) can distinguish them. The reasoning is that a calibrated law is fully determined by the score marginal \(F\), and for any such \(F\) one can honestly generate exactly that law by drawing a baseline covariate \(X\sim F\), sampling \(y\mid X=p\sim \mathrm{Bernoulli}(p)\), and reporting \(\hat p=X\). Therefore broad calibrated leakage is output-indistinguishable from honest performance unless an external \(\mathrm{AUC}_{\max}\) ceiling is supplied [2606.11267].

What leakage cannot hide is a near-deterministic subgroup. Sorting predictions \(p_{(1)}\ge \cdots \ge p_{(n)}\), the cumulative top-\(k\) purity is
\[
\rho(k)=\frac1k\sum_{i=1}^k y_{(i)},
\]
and the unit-purity head is
\[
s=\max\{k:\rho(k)\ge 1-\delta\},
\]
with slack \(\delta\in(0,1)\). The purity-ceiling lemma states that if the outcome is not prediction-time-deterministic, then every honest predictor must satisfy \(\rho(k)<1\) for all \(k\) that represent a non-null fraction of the population. A sustained region with \(\rho(k)\ge 1-\delta\) over \(k\ge k_{\min}\) therefore certifies near-deterministic leakage [2606.11267].

All detectors sort in \(O(n\log n)\) and then scan in \(O(n)\). The unified algorithm computes \(\rho(k)\), the spike head \(s\), the AUC \(c\), and a dispersion statistic
\[
V_\eta=
\frac{\sum_{i=1}^n w(p_i)^2 (y_i-p_i)^2}{\sum_{i=1}^n w(p_i)^2 p_i(1-p_i)},
\]
with \(w(p)=H(p)+G(p)\), and returns a verdict among clean or leaky together with a miscalibration warning. On UK Biobank with time-windowed comorbidity leakage of known graded severity, the measured detection floor was \(\Delta c^\star\approx 0.007\) on that endpoint; the paper emphasizes that this numerical floor is cohort- and endpoint-specific, whereas the structural lesson is general [2606.11267].

This yields a trichotomy. Miscalibrated leakage is detectable but evadable by recalibration. Broad-calibrated leakage requires an external discrimination ceiling. Deterministic or near-label leakage admits a prior-free detector. A plausible implication is that “leakage-free” cannot always be certified from outputs alone; in some regimes it is identifiable only through the data-generation and modeling protocol.

## 7. Internal-state interventions: speculative execution and reasoning traces

A broader systems perspective appears in work that prevents leakage by constraining the evolution of internal state rather than only the final prediction. In microarchitecture, ConTExT targets transient execution, where poisoned predictors or deferred faults allow secret data to influence microarchitectural side-effects. The specific structures considered are the Pattern History Table and Branch History Buffer, the Branch Target Buffer, the Return Stack Buffer, and store-to-load dependency speculation in the Reorder Buffer and Store Buffer. ConTExT’s principle is that secrets can enter registers, but not transiently leave them [1905.09100].

The mechanism is a co-design of minimal hardware extensions and small compiler/OS changes. A non-transient bit \(N\) is added to each page-table entry and TLB entry, one taint bit is added to each architectural register, and each data-cache line receives \(8\) extra bits to record register spills. Taint propagation follows
\[
T(r_1\leftarrow r_2):=T(r_2),
\]
\[
T(r\leftarrow M[a]):=A(a)\vee(\text{cache line taint}),
\]
\[
T(r\leftarrow op(x,y,\ldots)):=\bigvee_{\text{src}}T(\text{src})\vee\bigvee_{\text{src mem}}A(\mathrm{addr}(\text{src})).
\]
If a \(\mu\)-op is transient and any source has \(T=1\) or \(A=1\), the hardware forwards a canonical dummy value, such as zero, rather than the real secret. The resulting non-interference invariant is that \(\tau_{\mathrm{real}}(S)=\tau_{\mathrm{real}}(S')\) whenever \(S\) and \(S'\) differ only in non-transient pages. Reported overheads included \(71.14\%\pm 4.66\%\) on OpenSSL-RSA-encrypt under ConTExT-light, \(+48\) CPU cycles per syscall, \(0.15\,\mathrm{ms}\) per process startup, and \(<1\%\) slow-down in Bochs-simulated full ConTExT on realistic mixed workloads [1905.09100].

In large language models, the leakage target shifts from microarchitectural side-effects to reasoning traces. SALT addresses contextual privacy leakage in chain-of-thought by steering hidden activations away from “leaky” directions via a single additive edit at test time. For an input \(x\), leakage is measured by an indicator \(\ell(x)\) for whether the reasoning trace reveals inappropriate private details, and the Contextual Privacy Leakage metric is
\[
\mathrm{CPL}=\frac1N\sum_{i=1}^N \ell(x_i).
\]
Utility is measured by
\[
\mathrm{MOU}=\frac1N\sum_{i=1}^N u(x_i),
\]
where \(u(x)\) indicates a correct or coherent final answer. High-leakage layers are identified via Cohen’s \(d\),
\[
d_{t,j}=
\frac{\mu^{\mathrm{leak}}_{t,j}-\mu^{\mathrm{non}}_{t,j}}
{\sqrt{\tfrac12(\sigma^{2,\mathrm{leak}}_{t,j}+\sigma^{2,\mathrm{non}}_{t,j})}},
\]
and layer density
\[
\rho_t(\tau)=\frac1d\bigl|\{j:|d_{t,j}|\ge \tau\}\bigr|.
\]
For each layer, the steering vector is the normalized mean-difference \(\hat S_t\), and at inference
\[
H'_t(t^\star)=H_t(t^\star)+\lambda \hat S_t,
\qquad t=L_{\mathrm{last}}.
\]
Across QwQ-32B, Llama-3.1-8B, and DeepSeek-R1-Distill-Qwen-1.5B, leakage rose in the final \(20\)–\(30\%\) of blocks, peaking a few layers before the output head. SALT reduced CPL from \(0.727\) to \(0.595\) on QwQ-32B, from \(0.385\) to \(0.316\) on Llama-8B, and from \(0.077\) to \(0.053\) on DeepSeek-1.5B, with corresponding MOU changes from \(0.812\) to \(0.843\), \(0.758\) to \(0.710\), and \(0.106\) to \(0.109\) [2511.07772].

These systems differ in threat model and mechanism, but they share a common architectural intuition. Leakage is controlled by modifying the trajectory of hidden state itself: taint bits and dummy forwarding in a speculative processor, or activation steering at a selected layer and token in a transformer. This suggests that one important meaning of leakage-free state prediction is not merely output sanitization, but intervention on the internal pathways by which state becomes predictive or externally observable.

Source: https://www.emergentmind.com/topics/leakage-free-state-prediction