Selective Representation Space (SRS) Overview

• Selective Representation Space (SRS) is a learned representation that explicitly filters features via selectivity, balance, and predictivity for enhanced modeling.
• In causal inference, SRS employs deep feature selection with elastic net regularization and Wasserstein balance to achieve accurate treatment effect estimation through matching.
• In time series forecasting, SRS uses selective patching and dynamic reassembly to extract and reorder informative subseries, yielding improved long-horizon prediction performance.

Selective Representation Space (SRS) denotes a learned representation space whose content is explicitly filtered rather than fixed. In the cited literature, the term has two distinct technical uses. In treatment effect estimation, SRS is the image $Z = f_\theta(X \odot s)$ learned by Feature Selection Representation Matching (FSRM), where selectivity is induced by sparse feature gating, balance is enforced by a Wasserstein regularizer, and treatment effects are estimated by matching in the learned space (Chu et al., 2020). In patch-based time series forecasting, SRS is a plug-and-play module that replaces fixed adjacent patching with Selective Patching and Dynamic Reassembly, so that each input series is represented by a sample-specific set of informative patches before forecasting (Wu et al., 16 Oct 2025).

1. Terminological scope

The two uses of SRS are methodologically distinct, even though both center on learning a task-adaptive representation space rather than accepting a fixed input parameterization. In causal inference, the space is constructed from covariates and is required to be selective, balanced across treatment groups, and predictive of outcomes. In time series forecasting, the space is constructed from candidate subseries patches and is required to be selective with respect to forecasting utility and flexible with respect to patch order.

Context	Definition of SRS	Core mechanism
Treatment effect estimation	$Z = f_\theta(X \odot s)$	Deep feature selection, Wasserstein balance, matching
Time series forecasting	Sample-wise selective patch representation	Selective Patching, Dynamic Reassembly, Adaptive Fusion

A common source of confusion is to treat these two formulations as variants of the same architecture. They are not. The causal formulation is embedded in the potential outcomes framework and culminates in a matching estimator, whereas the forecasting formulation is a module at the patch-embedding interface of patch-based forecasters.

2. Causal-inference formulation of SRS

Within FSRM, the causal setting follows the potential outcomes framework with binary treatment. For each unit $i$ with covariates $X_i \in \mathbb{R}^d$ and treatment indicator $T_i \in \{0,1\}$, the potential outcomes are $Y_i(1)$ and $Y_i(0)$, while only the factual outcome $Y_i = Y_i(T_i)$ is observed. The target estimands are the average treatment effect $\mathrm{ATE} = \mathbb{E}[Y(1)-Y(0)]$, the average treatment effect on the treated $\mathrm{ATT} = \mathbb{E}[Y(1)-Y(0)\mid T=1]$, the conditional average treatment effect $Z = f_\theta(X \odot s)$0, and the individual treatment effect $Z = f_\theta(X \odot s)$1. Identification relies on SUTVA and consistency, ignorability, and positivity (Chu et al., 2020).

The learned SRS is defined by a representation mapping $Z = f_\theta(X \odot s)$2 together with an explicit feature-selection gate $Z = f_\theta(X \odot s)$3 applied element-wise to the inputs. The selective representation for an input $Z = f_\theta(X \odot s)$4 is

$Z = f_\theta(X \odot s)$5

Its design is governed by three requirements. Selectivity attenuates or suppresses irrelevant or harmful covariates, including pure instruments and noise. Balance aligns the distributions of treated and control representations so that overlap is improved and the learned space approximates a randomized setting. Predictivity retains sufficient information to predict outcomes under both treatments while also predicting treatment assignment.

FSRM realizes this design through a deep feature selection module, a treatment prediction head, an outcome prediction head with TARNet-style bifurcation, and a balance regularizer. The one-to-one feature selection layer has a diagonal weight matrix with diagonal $Z = f_\theta(X \odot s)$6, and deeper fully connected layers further shape selectivity. Sparsity and stability are induced by an elastic net penalty across representation layers:

$Z = f_\theta(X \odot s)$7

with $Z = f_\theta(X \odot s)$8 controlling smoothness and sparsity. This formulation is intended to retain correlated groups of features when needed, while still selecting a subset of inputs.

The prediction terms are a binary cross-entropy loss for treatment prediction,

$Z = f_\theta(X \odot s)$9

and a mean squared error on factual outcomes,

$i$0

where $i$1 and $i$2. Balance is enforced with the Wasserstein-1 distance between the treated and control representation distributions $i$3 and $i$4:

$i$5

The full objective jointly optimizes selectivity, predictivity, and balance:

$i$6

3. Matching, estimation, and empirical behavior in FSRM

After training, FSRM freezes $i$7 and $i$8, computes $i$9 for all units, and performs matching in $X_i \in \mathbb{R}^d$0. The matching distance can be Euclidean, Mahalanobis, or a distance in propensity-score space; in FSRM, Euclidean distance in $X_i \in \mathbb{R}^d$1 generally performed best. Matching may be nearest-neighbor matching or optimal bipartite matching, with or without replacement. For a unit $X_i \in \mathbb{R}^d$2, the matched opposite-treatment neighbor is

$X_i \in \mathbb{R}^d$3

The one-to-one CATE estimate is

$X_i \in \mathbb{R}^d$4

and the ATT estimator is

$X_i \in \mathbb{R}^d$5

ATE is then obtained by averaging $X_i \in \mathbb{R}^d$6 across all units, or by averaging group-specific estimators appropriately (Chu et al., 2020).

This formulation sits in direct relation to prior representation-learning approaches for causal inference. TARNet learns a shared representation with two outcome heads but has no explicit balance regularization and no feature selection. CFRNet augments TARNet with an IPM penalty, using MMD or Wasserstein, between treated and control representations. FSRM adds deep feature selection and uses matching in the learned space rather than relying only on regression at inference time. Other neural approaches, including Dragonnet-style objectives, often include propensity modeling and targeted regularization; FSRM explicitly includes a treatment head $X_i \in \mathbb{R}^d$7 and uses its joint training to shape a selective, prediction-relevant representation before matching.

The reported experiments use IHDP, modified IHDP, and a synthetic dataset. The metrics are $X_i \in \mathbb{R}^d$8 and $X_i \in \mathbb{R}^d$9. Baselines include kNN matching, causal forests, random forest, BART, GANITE, propensity score matching, TARNet, CFRNet (Wasserstein), SITE, and Perfect Match. FSRM with Euclidean distance consistently achieved the lowest ATE error across all datasets, with reported values of approximately $T_i \in \{0,1\}$0 on IHDP, $T_i \in \{0,1\}$1 on modified IHDP, and $T_i \in \{0,1\}$2 on the synthetic dataset. For ITE accuracy, FSRM was strongest on modified IHDP and the synthetic dataset, with reported $T_i \in \{0,1\}$3 values of approximately $T_i \in \{0,1\}$4 and $T_i \in \{0,1\}$5, respectively. On IHDP, where all variables are relevant by construction, some regression-based baselines had lower PEHE, which is consistent with the paper’s discussion of selectivity trade-offs.

The ablations are central to the interpretation of SRS in this setting. Removing feature selection substantially degraded performance; on the synthetic data, $T_i \in \{0,1\}$6 rose from $T_i \in \{0,1\}$7 to $T_i \in \{0,1\}$8. Removing the IPM balance term also hurt performance, with the degradation becoming more pronounced as treatment selection bias was amplified. Robustness experiments varying the artificial selection bias parameter $T_i \in \{0,1\}$9 from $Y_i(1)$0 to $Y_i(1)$1 showed maintained superiority, indicating robustness of the balanced SRS. At the same time, the method retains the usual limitations of observational causal inference: if important confounders are unobserved, bias remains; when all covariates are relevant, aggressive feature selection may discard useful signal; and the paper does not provide new formal guarantees specific to FSRM’s matching estimator.

4. Forecasting formulation of SRS

In patch-based time series forecasting, SRS is a plug-and-play module designed to replace the fixed, adjacent patching paradigm with a learnable, data-dependent mechanism that selectively chooses and reorders the most informative patches for each input series. The motivation is that conventional adjacent patching partitions a contextual time series into contiguous patches with fixed stride, thereby imposing a single, rigid representation space across all inputs. The paper identifies changeable periods, shifting, and anomalies as regimes in which fixed adjacency can break periodic semantics or drag harmful segments into the representation, degrading long-horizon forecasting (Wu et al., 16 Oct 2025).

For a univariate contextual time series $Y_i(1)$2, with patch size $Y_i(1)$3 and stride $Y_i(1)$4, the standard adjacent patch extraction operator forms

$Y_i(1)$5

patches, with $Y_i(1)$6 and

$Y_i(1)$7

SRS instead considers all candidate patches at stride $Y_i(1)$8. If the backbone expects $Y_i(1)$9 patches at stride $Y_i(0)$0, then the padded length is $Y_i(0)$1, giving

$Y_i(0)$2

candidate start indices and candidate patch tensor $Y_i(0)$3.

Selective Patching scores each candidate patch with a learnable scorer $Y_i(0)$4:

$Y_i(0)$5

where $Y_i(0)$6. For each of the $Y_i(0)$7 sampling steps, the module selects the patch index with the highest score, allowing selection with replacement. Because argmax is non-differentiable, SRS uses a straight-through construction:

$Y_i(0)$8

$Y_i(0)$9

The forward selection remains hard, but gradients propagate through the scorer.

Dynamic Reassembly then learns an order over the selected patches. A second scorer $Y_i = Y_i(T_i)$0 assigns one score to each selected patch:

$Y_i = Y_i(T_i)$1

After sorting and applying the same straight-through trick,

$Y_i = Y_i(T_i)$2

$Y_i = Y_i(T_i)$3

yielding a learned permutation over the $Y_i = Y_i(T_i)$4 selected patches. The paper notes that the implementation uses index-based sorting plus this straight-through trick rather than Sinkhorn relaxations.

The final representation fuses the conventional adjacent-patch embedding and the SRS-selected embedding:

$Y_i = Y_i(T_i)$5

$Y_i = Y_i(T_i)$6

with $Y_i = Y_i(T_i)$7. In the multivariate setting, the method operates channel-independently on $Y_i = Y_i(T_i)$8, producing $Y_i = Y_i(T_i)$9. The combined search space of Selective Patching and Dynamic Reassembly has size

$\mathrm{ATE} = \mathbb{E}[Y(1)-Y(0)]$0

reflecting selection with replacement and all permutations over the selected patches.

5. SRSNet, plugin-and-play integration, and empirical behavior

SRSNet is the minimal forecasting architecture built around this SRS module: SRS constructs the selective representation space, and a $\mathrm{ATE} = \mathbb{E}[Y(1)-Y(0)]$1–$\mathrm{ATE} = \mathbb{E}[Y(1)-Y(0)]$2 layer MLP head maps the resulting embeddings to the forecast. The forward pipeline consists of Instance Normalization, adjacent patch extraction, stride-$\mathrm{ATE} = \mathbb{E}[Y(1)-Y(0)]$3 candidate extraction, Selective Patching, Dynamic Reassembly, embedding and fusion, positional encoding, flattening, and MLP prediction. Training is end-to-end with the standard point-forecasting mean squared error,

$\mathrm{ATE} = \mathbb{E}[Y(1)-Y(0)]$4

while mean absolute error is used for evaluation as well. No extra regularizers for selection or permutation are required, because gradients flow through the straight-through masks in the selection and sorting stages (Wu et al., 16 Oct 2025).

The computational overhead is described as favorable. Candidate patch extraction has $\mathrm{ATE} = \mathbb{E}[Y(1)-Y(0)]$5 time and memory; scoring for selection is dominated by the scorer MLP over the $\mathrm{ATE} = \mathbb{E}[Y(1)-Y(0)]$6 candidates; Dynamic Reassembly adds scorer cost plus an $\mathrm{ATE} = \mathbb{E}[Y(1)-Y(0)]$7 sort; and the additional embedding pass is linear in $\mathrm{ATE} = \mathbb{E}[Y(1)-Y(0)]$8. When SRS is plugged into PatchTST and Crossformer, memory and time increase by about $\mathrm{ATE} = \mathbb{E}[Y(1)-Y(0)]$9 and MACs by less than $\mathrm{ATT} = \mathbb{E}[Y(1)-Y(0)\mid T=1]$0, reported as $\mathrm{ATT} = \mathbb{E}[Y(1)-Y(0)\mid T=1]$1–$\mathrm{ATT} = \mathbb{E}[Y(1)-Y(0)\mid T=1]$2 memory/time and $\mathrm{ATT} = \mathbb{E}[Y(1)-Y(0)\mid T=1]$3–$\mathrm{ATT} = \mathbb{E}[Y(1)-Y(0)\mid T=1]$4 MACs. The practical recommendation is to use small $\mathrm{ATT} = \mathbb{E}[Y(1)-Y(0)\mid T=1]$5, such as $\mathrm{ATT} = \mathbb{E}[Y(1)-Y(0)\mid T=1]$6–$\mathrm{ATT} = \mathbb{E}[Y(1)-Y(0)\mid T=1]$7, and modest patch lengths such as $\mathrm{ATT} = \mathbb{E}[Y(1)-Y(0)\mid T=1]$8–$\mathrm{ATT} = \mathbb{E}[Y(1)-Y(0)\mid T=1]$9.

The empirical evaluation covers eight multivariate benchmarks: ETTh1, ETTh2, ETTm1, ETTm2, Weather, Electricity, Solar, and Traffic, using MSE and MAE at forecast horizons $Z = f_\theta(X \odot s)$00. SRSNet achieves state-of-the-art average performance across these datasets. Reported average MSE/MAE values include ETTh1 $Z = f_\theta(X \odot s)$01, ETTh2 $Z = f_\theta(X \odot s)$02, ETTm2 $Z = f_\theta(X \odot s)$03, Electricity $Z = f_\theta(X \odot s)$04, Solar $Z = f_\theta(X \odot s)$05, and Traffic $Z = f_\theta(X \odot s)$06. On Weather, SRSNet is reported as near-best in MSE with $Z = f_\theta(X \odot s)$07, while the best MSE is $Z = f_\theta(X \odot s)$08 by Amplifier. On ETTm1, it reaches $Z = f_\theta(X \odot s)$09, tying the best MAE and remaining near-best in MSE.

The ablations attribute the largest performance drop to removing Selective Patching, indicating that the decision of what to include is the principal driver. Removing Dynamic Reassembly or Adaptive Fusion also degrades results, indicating that ordering and fusion each contribute. The module is also reported as plugin-and-play across patch-based backbones. Adding SRS to a plain MLP improves MSE by about $Z = f_\theta(X \odot s)$10–$Z = f_\theta(X \odot s)$11 across ETTh1, ETTm2, Solar, and Traffic. Adding SRS to PatchTST, Crossformer, xPatch, and PatchMLP yields approximately $Z = f_\theta(X \odot s)$12–$Z = f_\theta(X \odot s)$13 average MSE reductions, with PatchTST gains of $Z = f_\theta(X \odot s)$14–$Z = f_\theta(X \odot s)$15 depending on dataset and horizon, and Crossformer improvements exceeding $Z = f_\theta(X \odot s)$16 on ETTm2.

The implementation guidance is specific. SRS is inserted at the patch embedding interface, where conventional patch-tokenization is replaced or augmented by a dual-path embedding of adjacent patches and SRS-selected patches followed by fusion. Typical settings retain the backbone’s patch size $Z = f_\theta(X \odot s)$17 and stride $Z = f_\theta(X \odot s)$18, set $Z = f_\theta(X \odot s)$19, use scorer MLPs with $Z = f_\theta(X \odot s)$20 hidden layers and hidden size $Z = f_\theta(X \odot s)$21, and initialize $Z = f_\theta(X \odot s)$22 near $Z = f_\theta(X \odot s)$23, or slightly higher for more stationary periodic datasets and slightly lower for more shifting series. The limitations are equally explicit: SRS assumes a patch-based backbone; forcing patch size $Z = f_\theta(X \odot s)$24 for non-patch models is possible but not advisable; selected patches are useful but not guaranteed to be human-interpretable; and scaling behavior in billion-sample pretraining remains to be validated.

6. Comparative interpretation and recurrent misconceptions

The two SRS formulations share a reliance on learnable selectivity, but they differ in objective, geometry, and downstream estimator. In the causal setting, SRS is a low-dimensional space over covariates in which irrelevant, instrumental, and noisy variables are attenuated, treated and control distributions are aligned, and matching is carried out after training. In the forecasting setting, SRS is a patch-selection and reordering mechanism over contextual subseries, optimized directly by forecasting loss before any downstream head (Chu et al., 2020, Wu et al., 16 Oct 2025).

Several misconceptions are addressed directly by the cited work. In causal inference, balance in representation space does not remove the need for ignorability and overlap; if important confounders are unobserved, bias remains. Nor does selectivity guarantee uniformly better individual treatment estimates: when all covariates are relevant, aggressive feature selection can slightly harm ITE performance. In forecasting, SRS is not an interpretability module, even though it selects patches; the selected patches are not guaranteed to be human-interpretable. It is also not a universal tokenization scheme for arbitrary architectures, because the formulation is designed for patch-based backbones.

Taken together, the two bodies of work suggest a broader interpretation of SRS as a task-specific learned space in which representational capacity is allocated selectively rather than uniformly. In one case, selectivity is coupled to distributional balance and matching for counterfactual estimation. In the other, selectivity is coupled to sample-wise patch inclusion and learned ordering for long-horizon forecasting. The shared label therefore identifies a common design principle, but not a single standardized method.