Transductive Online Regression

• Transductive online regression is a framework that optimizes predictions by exploiting the entire realized input set, integrating full-sequence, active, and context–query protocols.
• It employs methods such as cover-based multiplicative weights, thresholding for active label selection, and online Lewis weight sampling to achieve near-optimal regret bounds and reduced prediction error.
• The framework's applications span linear regression, collaborative filtering, and operator learning, underscoring its theoretical rigor and practical effectiveness in sequential prediction tasks.

Transductive online regression denotes a family of sequential regression settings in which the target of optimization is tied to a realized collection of inputs—an arriving stream, a fixed unlabeled sequence, or a specified query set—rather than to an abstract future test distribution. In one line of work, the learner is given the entire unlabeled sequence $x_{1:T}$ before prediction and is evaluated by regret against a function class on that sequence (Raman et al., 4 Oct 2025). In another, covariates arrive online and the learner decides which labels to query under a budget while optimizing the empirical regression objective on the received points (Chen et al., 2022, Riquelme et al., 2016). A third usage treats transduction as direct prediction at query inputs from a context set, often with no task-specific retraining at inference time (Chalvidal et al., 2023). This suggests that the term is best understood as an umbrella for online regression protocols in which the realized inputs themselves play a privileged algorithmic role.

1. Core formulations and protocol variants

The literature uses closely related but non-identical protocol definitions.

Formulation	Information pattern	Primary objective
Transductive online learning	Full unlabeled sequence $x_{1:T}$ revealed before prediction	Minimize regret on that sequence
Online active regression	Rows arrive sequentially; label query decisions are irrevocable	Approximate the empirical regression objective with few queried labels
Context–query transduction	Context set $C$ and query set $Q$ are given	Predict outputs at the specified queries

In the transductive online learning protocol, the learner sees $x_t$, predicts $\hat y_t$, then observes $y_t$; the transductive feature is that the entire unlabeled sequence $x_{1:T}$ is known before the game starts, so prediction at round $t$ may depend on $(x_{1:t},y_{1:t-1},x_{t+1:T})$ (Raman et al., 4 Oct 2025). A closely related setup assumes that the learner is given the full multiset of design vectors $x_{1:T}$0 before the game starts, though their order is hidden; this formulation is used to build design-dependent priors for regression with unbounded losses (Qian et al., 2024).

In online active regression, by contrast, rows $x_{1:T}$1 arrive one by one, each with an unknown label $x_{1:T}$2, and the learner must immediately decide whether to query the label, subject to a global budget $x_{1:T}$3 (Chen et al., 2022). The objective is transductive because the algorithms optimize the regression objective over the points that actually arrived, with no explicit generalization analysis to future unseen samples (Chen et al., 2022). In the statistical linear-model formulation, the goal is to minimize expected mean squared prediction error on the encountered random-design points under a labeling budget $x_{1:T}$4, which reduces to minimizing $x_{1:T}$5 over the queried set $x_{1:T}$6 (Riquelme et al., 2016).

A separate function-space formulation treats transductive regression as direct estimation at query inputs from a finite context set. Given $x_{1:T}$7 and queries $x_{1:T}$8, a transducer $x_{1:T}$9 outputs $C$0 by aggregating context information in a permutation-invariant manner (Chalvidal et al., 2023). This formulation is online and transductive in the sense that adding or removing context pairs changes the forward pass directly, without task-specific retraining.

A recurrent point of confusion is the relation to fixed-design denoising. The transductive-prior literature explicitly distinguishes sequential transductive regression from standard denoising: the learner exploits the full horizon of design vectors but still predicts sequentially, and this sequential structure enables online-to-batch conversion results that are described as impossible for standard denoising results (Qian et al., 2024).

2. Minimax theory, complexity measures, and separations

For full-sequence transductive online regression, the central benchmark is minimax regret against a hypothesis class $C$1 under a convex $C$2-Lipschitz loss. The transductive regret of a learner $C$3 is

$C$4

with the crucial difference from adversarial online learning being that the unlabeled sequence is revealed before predictions begin (Raman et al., 4 Oct 2025).

The 2025 minimax characterization shows that transductive regret is governed by the non-sequential fat-shattering dimension, not the sequential one. The upper bound takes the form

$C$5

where $C$6 is a Rademacher complexity on the fixed revealed sequence, and this is further controlled by an entropy integral involving $C$7. For $C$8-loss, the lower bound is

$C$9

Together these results establish that fat-shattering “fully governs” the transductive minimax expected regret, up to constants and logarithmic factors (Raman et al., 4 Oct 2025).

This yields a strict separation from adversarial online regression. The adversarial setting is controlled by sequential fat-shattering; many natural classes have finite non-sequential fat-shattering dimension but infinite sequential fat-shattering dimension, and are therefore transductively learnable but not adversarially online learnable (Raman et al., 4 Oct 2025). The bounded-variation class $Q$0 is a canonical example: it satisfies $Q$1, giving $Q$2, while infinite sequential fat-shattering precludes adversarial online learnability (Raman et al., 4 Oct 2025).

A constructive algorithmic counterpart is a cover-based multiplicative weights procedure. Given the revealed sequence $Q$3, one builds an $Q$4-cover $Q$5 of the traces $Q$6, treats each cover element as an expert, and runs MWA. The resulting bound is

$Q$7

where $Q$8 (Raman et al., 4 Oct 2025). The same paper extends the framework to learning-augmented online regression, where noisy predictions of future examples induce interpolation bounds between transductive and worst-case adversarial regret.

An earlier computational strand approached the same transductive setting through random playout and randomized rounding. The “R$Q$9 Forecaster” assumes the full unlabeled sequence is known in advance, converts convex-loss subgradients into randomized binary labels, and reduces prediction to repeated ERM computations under random future outcomes (Cesa-Bianchi et al., 2011). Its high-probability regret is controlled by transductive Rademacher complexity, and the method gives an efficient online algorithm for collaborative filtering with trace-norm constrained matrices; in the square $x_t$0 case over horizon $x_t$1, the average per-round regret is

$x_t$2

for $x_t$3 (Cesa-Bianchi et al., 2011). This line of work also resolved an open batch-to-transductive question by showing that efficient ERM can imply efficient transductive online learning with $x_t$4-type rates (Cesa-Bianchi et al., 2011).

3. Budgeted transductive linear regression via thresholding

A statistically explicit version of transductive online regression arises in online active linear regression under random design. The model is

$x_t$5

with feature vectors $x_t$6 arriving sequentially and a labeling budget $x_t$7 (Riquelme et al., 2016). If $x_t$8 is the queried set and $x_t$9 is the resulting design matrix, then under random design

$\hat y_t$0

and the expected mean squared error in $\hat y_t$1-norm is

$\hat y_t$2

Thus the transductive objective reduces to choosing which encountered points to label so as to minimize $\hat y_t$3 (Riquelme et al., 2016).

The thresholding algorithm first whitens the covariates, then defines an informativeness statistic

$\hat y_t$4

where $\hat y_t$5 are weights. The rule is to query $\hat y_t$6 if $\hat y_t$7, with a “must fill budget” override when the remaining number of observations equals the remaining budget (Riquelme et al., 2016). The threshold is calibrated by

$\hat y_t$8

and for Gaussian white covariates with $\hat y_t$9, one has $y_t$0 and therefore $y_t$1 (Riquelme et al., 2016).

The design rationale is spectral: the method seeks to enlarge and balance the eigenvalues of the selected information matrix. Under the induced distribution conditioned on $y_t$2, the selected information matrix has eigenvalues determined by the conditional second moments $y_t$3, and balancing these moments improves both expected MSE and its variance (Riquelme et al., 2016). Under whitened symmetric subgaussian designs with $y_t$4, the main high-probability guarantee is

$y_t$5

In the Gaussian specialization with $y_t$6 and $y_t$7,

$y_t$8

whereas passive random sampling under white Gaussian design gives $y_t$9 (Riquelme et al., 2016). Matching lower bounds show that no selection algorithm can improve the order beyond the same log-enhanced structure in the Gaussian regime, so the thresholding rule is near-optimal there (Riquelme et al., 2016).

The paper also gives a sparse high-dimensional extension. Stage 1 labels the first $x_{1:T}$0 points, computes a Lasso estimate, and forms $x_{1:T}$1. Stage 2 restricts to the $x_{1:T}$2-dimensional subspace $x_{1:T}$3, applies thresholding there with $x_{1:T}$4, and then fits OLS on the selected second-stage points (Riquelme et al., 2016). Under Gaussian design and standard support recovery conditions, the bound becomes

$x_{1:T}$5

and the paper states that sparse thresholding can achieve gains at least by a $x_{1:T}$6 factor over passive sampling with Lasso when $x_{1:T}$7 and $x_{1:T}$8 (Riquelme et al., 2016).

Empirically, thresholding consistently reduced both mean and variance of MSE relative to passive sampling. On Protein Structure, Bike Sharing, and YearPredictionMSD, thresholding improved median, mean, and variance of test MSE, with gains most pronounced in moderate-budget regimes and diminishing as $x_{1:T}$9 grows large (Riquelme et al., 2016). The same study reports robustness even under high nonlinearity and misspecification, though it notes that very large $t$0 can push selection toward tail regions where nonlinearity becomes more harmful (Riquelme et al., 2016).

4. Sketching-based online active regression under $t$1 loss

A more adversarial and distribution-free formulation studies online active regression for $t$2 loss with $t$3. Here the objective is

$t$4

where rows $t$5 arrive sequentially, labels are costly, and the learner must maintain a near-optimal solution using only a queried subset of labels (Chen et al., 2022). All guarantees are transductive: they refer to the empirical objective on the received rows, not to future unseen data (Chen et al., 2022).

The key machinery is online Lewis-weight sampling. For $t$6, the Lewis weights $t$7 are defined implicitly by

$t$8

and the online Lewis weight $t$9 is the Lewis weight of $(x_{1:t},y_{1:t-1},x_{t+1:T})$0 with respect to the prefix $(x_{1:t},y_{1:t-1},x_{t+1:T})$1 (Chen et al., 2022). Monotonicity guarantees $(x_{1:t},y_{1:t-1},x_{t+1:T})$2, so sampling with probabilities proportional to online weights yields valid subspace embeddings in the online setting (Chen et al., 2022). The resulting diagonal sampling matrix $(x_{1:t},y_{1:t-1},x_{t+1:T})$3 preserves geometry: $(x_{1:t},y_{1:t-1},x_{t+1:T})$4 with high probability (Chen et al., 2022).

Three algorithmic variants are given. For $(x_{1:t},y_{1:t-1},x_{t+1:T})$5, the method maintains four sketches $(x_{1:t},y_{1:t-1},x_{t+1:T})$6, computes a constant-factor solution, forms a residual problem, and refines it on smaller subsamples (Chen et al., 2022). For $(x_{1:t},y_{1:t-1},x_{t+1:T})$7, Lewis weights reduce to leverage scores, which can be updated online via Sherman–Morrison rank-one updates, and a sparse Johnson–Lindenstrauss transform is used to accelerate computations (Chen et al., 2022). For $(x_{1:t},y_{1:t-1},x_{t+1:T})$8, a simpler single-sketch algorithm suffices and updates the sketched $(x_{1:t},y_{1:t-1},x_{t+1:T})$9 regression solution online (Chen et al., 2022).

The principal label-complexity statement is that to achieve a $x_{1:T}$00-approximate solution, the algorithms require only $x_{1:T}$01 label queries, where $x_{1:T}$02 is an online condition number of the stream (Chen et al., 2022). The explicit bounds depend on $x_{1:T}$03, $x_{1:T}$04, and either $x_{1:T}$05 or $x_{1:T}$06 in the $x_{1:T}$07 case, but the qualitative message is that transductive geometry preservation can be achieved with sublinear space and a label budget scaling essentially linearly in $x_{1:T}$08 up to polylogarithmic factors (Chen et al., 2022).

Computationally, exact online Lewis weights are too expensive, so the paper introduces a compression structure with $x_{1:T}$09 blocks $x_{1:T}$10, each a resampled summary of a chunk of the stream. This yields constant-factor approximations to the true online weights in $x_{1:T}$11 time per update for general $x_{1:T}$12 (Chen et al., 2022). For $x_{1:T}$13, leverage-score estimation and Sherman–Morrison updates give faster per-update behavior specialized to the quadratic case (Chen et al., 2022).

Empirically, the online methods significantly outperform online uniform sampling. On synthetic data and the Gas Sensor dataset, the paper reports that the online algorithm is significantly better than online uniform for $x_{1:T}$14, $x_{1:T}$15, and $x_{1:T}$16; for $x_{1:T}$17 it also outperforms the thresholding baseline from the earlier threshold-based active regression literature, while remaining comparable to offline active regression methods on several settings (Chen et al., 2022).

5. Transductive priors, unbounded losses, and drifting comparators

Another strand exploits foreknowledge of the design vectors to construct design-dependent priors for sequential prediction with unbounded losses. In this setup the full multiset of design vectors $x_{1:T}$18 is known before the game starts, though the order is unknown (Qian et al., 2024). Exponential weights is run with a Gaussian prior aligned to the Gram matrix $x_{1:T}$19,

$x_{1:T}$20

or with heavy-tailed sparse priors in the sparse case (Qian et al., 2024).

For squared-loss regression with $x_{1:T}$21 and invertible $x_{1:T}$22, the recovered transductive bound is

$x_{1:T}$23

With $x_{1:T}$24, this becomes

$x_{1:T}$25

a regret term depending only on $x_{1:T}$26, $x_{1:T}$27, and $x_{1:T}$28, not on $x_{1:T}$29 or $x_{1:T}$30 (Qian et al., 2024). For sparse regression, under a smallest scaled singular value condition with parameter $x_{1:T}$31, the sparse transductive estimator satisfies

$x_{1:T}$32

for any $x_{1:T}$33-sparse comparator $x_{1:T}$34 (Qian et al., 2024).

A notable consequence is an online-to-batch conversion in the transductive setup. If an algorithm has a deterministic transductive regret bound independent of the realized sequence, then the averaged predictor built from an i.i.d. sample and an additional unlabeled test point satisfies an expected excess-risk bound of order $x_{1:T}$35 (Qian et al., 2024). In particular, the squared-loss regression bound above yields

$x_{1:T}$36

with no assumptions on the distribution of $x_{1:T}$37 beyond measurability (Qian et al., 2024).

Non-stationary online regression provides a different extension. ARCOR and LASER are analyzed in the standard sequential protocol, but the paper explicitly notes that if the input sequence $x_{1:T}$38 is known beforehand, quantities such as $x_{1:T}$39 can be precomputed offline and the min-max predictor form remains unchanged (Vaits et al., 2013). ARCOR uses adaptive covariance resets and achieves

$x_{1:T}$40

under a suitable threshold schedule when the total variation drift $x_{1:T}$41 is sublinear, and it recovers logarithmic regret in the stationary case $x_{1:T}$42 (Vaits et al., 2013). LASER uses a last-step min-max derivation with quadratic drift regularization and, in the low-drift regime, yields a bound of order

$x_{1:T}$43

up to the explicit dimension-, norm-, and log-determinant terms given in the theorem, again reducing to logarithmic stationary regret when $x_{1:T}$44 (Vaits et al., 2013). The transductive effect here is primarily on preprocessing and constant optimization, not on the asymptotic regret rates.

6. Functional transduction, applications, and conceptual boundaries

In function-space and operator-learning settings, transductive online regression is instantiated by meta-learned context-to-query prediction. The Transducer model takes a context set $x_{1:T}$45 and a query $x_{1:T}$46, and outputs $x_{1:T}$47 in a single feedforward pass, with no task-specific gradient descent (Chalvidal et al., 2023). The construction is grounded in vector-valued Reproducing Kernel Banach Space theory: the learned operator-valued kernel $x_{1:T}$48 supports expansions of the form

$x_{1:T}$49

and the architecture realizes this through multi-head operator-valued attention and residual kernel stacking (Chalvidal et al., 2023).

The meta-learning objective is episodic: tasks are sampled from a distribution, split into context and query subsets, and the parameters $x_{1:T}$50 are optimized so that the feedforward transductive map minimizes the task loss over the query points (Chalvidal et al., 2023). Because the context is treated as a set, the architecture is permutation invariant; because the kernel is operator valued, outputs may be infinite-dimensional and represented in a spectral basis (Chalvidal et al., 2023). The forward complexity is approximately

$x_{1:T}$51

which is quadratic in the combined context-query size and fully parallelizable on GPUs (Chalvidal et al., 2023).

The reported applications are broad. For the Advection–Diffusion–Reaction PDE task with $x_{1:T}$52, the Transducer achieved regression time $x_{1:T}$53 and $x_{1:T}$54 GFLOPs per operator, versus $x_{1:T}$55 and $x_{1:T}$56 GFLOPs for FNO trained from scratch; the reported RMSEs were $x_{1:T}$57 for the Transducer and $x_{1:T}$58 for FNO (Chalvidal et al., 2023). The same work reports robust out-of-distribution behavior in PDE tasks, 100% precision/recall for a Burgers-equation outlier-detection experiment based on bootstrapped transductive fits, and improved latitude-weighted MSE in a Transducer–ViT climate model relative to a ViT baseline (Chalvidal et al., 2023).

Across the literature, applications reflect the underlying protocol. Threshold-based active linear regression is evaluated on Protein Structure, Bike Sharing, and YearPredictionMSD, where reductions in mean and variance of test MSE are reported (Riquelme et al., 2016). Sketch-based online active regression is tested on synthetic data and the Gas Sensor dataset, where it is significantly better than online uniform sampling and competitive with offline active regression (Chen et al., 2022). Random-planning transductive online learning has an application to collaborative filtering with trace-norm constraints (Cesa-Bianchi et al., 2011). Design-dependent priors yield assumption-free random-design statistical consequences for linear regression with squared loss (Qian et al., 2024).

Several misconceptions recur. One is that transductive online regression is synonymous with active label querying; the full-sequence regret literature and the transductive-prior literature show that label querying is only one branch of the topic (Raman et al., 4 Oct 2025, Qian et al., 2024). Another is that transductive access automatically implies a standard fixed-design batch problem; the sequential-transductive analyses explicitly reject that identification (Qian et al., 2024). A third is that stronger knowledge of inputs uniformly improves performance across all regimes: the active thresholding results note that gains tend to diminish when the label budget becomes large, because informational outliers may cease to align with prediction error on the realized test distribution (Riquelme et al., 2016).

Taken together, the literature presents transductive online regression as a spectrum of methods that exploit realized inputs more aggressively than inductive online regression does. The exploitation may take the form of minimax planning over a revealed sequence, budget-aware label selection on an arriving stream, design-dependent priors matched to a known Gram matrix, or direct kernelized construction of query predictions from a context set. What unifies these approaches is not a single algorithmic template, but the decision to optimize for the specific inputs at hand.