Intermediate Feature Leakage

• Intermediate feature leakage is the unintended retention of extra information in model intermediates, posing risks to privacy and interpretability.
• It appears in various settings including transfer learning, split-DNN, and federated learning, where intermediate activations can reveal sensitive details.
• Mitigation strategies such as adversarial training, noise injection, and architectural modifications are employed to restrict unwanted information flow.

Intermediate feature leakage denotes the persistence or transmission of unintended information in internal representations that are nominally introduced as compressed, task-relevant, or interpretable intermediates. In the cross-hospital chest X-ray setting, it means that frozen self-supervised features still encode hospital identity strongly enough that a simple post hoc classifier can recover site labels (Bouaziz et al., 31 Mar 2026). In split-DNN, split-learning, and vertical federated learning pipelines, it refers to the ability of intermediate activations or embeddings to reveal private inputs, semantics, or labels (Xiu et al., 30 Jul 2025, Arif et al., 8 Apr 2025, Roth et al., 2022, Sun et al., 2022, Ren et al., 19 Nov 2025). In concept-based models, it denotes label-relevant or interconcept information that bypasses the intended semantic bottleneck (Parisini et al., 18 Apr 2025, Makonnen et al., 13 Apr 2025, Ragkousis et al., 2024). A broader reading across these literatures is that leakage is not a single failure mode but a family of representational pathologies in which an intermediate state remains too informative for the role it is supposed to play.

1. Conceptual definitions and scope

A common operational definition appears in cross-hospital representation learning. The chest X-ray transfer study defines leakage as how easily acquisition site can be predicted from frozen features, distinguishing backbone features

$h = f_\theta(x) \in \mathbb{R}^{512}$

from projection-head features

$z = \mathrm{norm}(p_\psi(h)) \in \mathbb{R}^{d}.$

If a linear probe can still predict hospital or site from $h$ or $z$, then the representation still contains site leakage; the paper is explicit that this is not a full mutual-information estimate, but a practical, interpretable leakage metric based on probe accuracy (Bouaziz et al., 31 Mar 2026).

In concept bottleneck models, leakage is defined more explicitly as extra label-relevant information encoded in the learned concept representation beyond what is present in the intended concepts. One formulation uses conditional mutual information,

$I(y;\hat{c}\mid c)=H(y\mid c)-H(y\mid \hat{c},c),$

so that leakage is the reduction in label uncertainty provided by the learned bottleneck $\hat{c}$ after the ground-truth concepts $c$ are already known (Makonnen et al., 13 Apr 2025). A complementary information-theoretic treatment separates concepts-task leakage (CTL) from interconcept leakage (ICL), arguing that learned concepts may carry excess information both about the task label and about other concepts (Parisini et al., 18 Apr 2025).

In privacy-oriented systems, the same idea is instantiated at the protocol level. CapRecover treats intermediate visual features $\mathbf{F}=\mathcal{V}_{image}(I)$ as leaked objects that preserve object identity, scene context, and caption-level relations even when no pixel reconstruction is performed (Xiu et al., 30 Jul 2025). The vertical federated learning study defines the cut-layer embedding $f(X)$ as a potential label-bearing representation whose correlation with private labels enables label inference from the forward pass (Sun et al., 2022). Split-U-Net similarly treats transmitted activation maps and returned gradients as tensors that may encode enough information to reconstruct biomedical images or reveal labels (Roth et al., 2022).

A distinct but related temporal usage appears in student analytics. There, intermediate feature leakage denotes the accidental use of information that lies between the observation point and the outcome horizon, or even after the horizon, when constructing predictors. The CAPIRE data layer uses the Value of Observation Time (VOT) precisely to prevent such temporally intermediate information from entering the feature space (Paz, 14 Nov 2025). This suggests that “intermediate” can refer either to representational depth inside a model or to temporal position inside a predictive pipeline.

2. Measurement and quantification

The most direct measurement strategy is post hoc probing. In the chest X-ray study, leakage is measured by freezing the pretrained encoder, extracting $h$ and/or $z = \mathrm{norm}(p_\psi(h)) \in \mathbb{R}^{d}.$0 from a balanced dataset, training multinomial logistic regression to predict acquisition site, and evaluating on a held-out split. For the main binary setting, the sites are NIH versus CheXpert, chance accuracy is $z = \mathrm{norm}(p_\psi(h)) \in \mathbb{R}^{d}.$1, the probe uses a balanced sample of 7000 images per site, and the same train/test splits are shared across methods; a three-site probe for NIH versus CheXpert versus RSNA is also reported, where chance is $z = \mathrm{norm}(p_\psi(h)) \in \mathbb{R}^{d}.$2 (Bouaziz et al., 31 Mar 2026).

A second family of measurements uses inversion quality. PEEL evaluates recovery of inputs from residual-network block outputs with MSE and PSNR, and reports Attack Accuracy and KNN Distance when comparing against generative inversion baselines (Arif et al., 8 Apr 2025). Split-U-Net quantifies leakage by the Structural Similarity Index,

$z = \mathrm{norm}(p_\psi(h)) \in \mathbb{R}^{d}.$3

between the reconstructed image and the original image, because SSIM is treated as more perceptually meaningful than RMSE or PSNR for judging whether an inversion reveals anatomical content (Roth et al., 2022). FIA-Flow extends this measurement family with PSNR, SSIM, LPIPS, top-1 classification accuracy of reconstructions, and two large-vision-language-model-based metrics, LVLM-C and LVLM-PL, to capture whether the reconstructed image preserves primary-object identity and semantic content from a human perspective (Ren et al., 19 Nov 2025).

A third family measures semantic or label leakage without reconstructing pixels. CapRecover maps leaked visual features directly to labels or captions and reports Top-1 and Top-5 label accuracy, BLEU-1, ROUGE-L, and cosine-similarity success. On CIFAR-10 it reaches up to 92.71% Top-1 label accuracy, and on COCO2017 it generates captions with ROUGE-L scores up to 0.52, showing that intermediate features can leak high-level semantics directly (Xiu et al., 30 Jul 2025). In vertical federated learning, leakage is quantified by Leak AUC, where values near $z = \mathrm{norm}(p_\psi(h)) \in \mathbb{R}^{d}.$4 indicate strong label leakage from forward embeddings and values near $z = \mathrm{norm}(p_\psi(h)) \in \mathbb{R}^{d}.$5 indicate random guessing (Sun et al., 2022).

Concept-based methods add information-theoretic leakage scores. The CTL metric compares the mutual information of learned versus true concepts with the task label, while ICL compares learned versus true interconcept dependence; these measures are presented as more robust and reliable than alternatives such as OIS and NIS for predicting intervention behavior (Parisini et al., 18 Apr 2025). Tree-Based Leakage Inspection and Control in CBMs localizes leakage by approximating

$z = \mathrm{norm}(p_\psi(h)) \in \mathbb{R}^{d}.$6

with decision-tree information gain inside a hard-tree leaf, so that a leaky split is one where a soft concept probability reduces label entropy beyond what the hard concept path already explains (Ragkousis et al., 2024).

In scientific modeling outside the usual privacy setting, the CA1 hippocampal MaxEnt study introduces diagnostics that separate in-distribution prediction from mechanism recovery: state reweighting that changes $z = \mathrm{norm}(p_\psi(h)) \in \mathbb{R}^{d}.$7 while holding $z = \mathrm{norm}(p_\psi(h)) \in \mathbb{R}^{d}.$8 fixed, conditional log-odds contrasts for local additivity, and temporal leakage controls. Its central point is that direct entropy explained is a predictive summary under the sampled input distribution, not a mechanism-identification test (Safaai et al., 1 Jun 2026).

3. Mechanisms that produce leakage

One recurrent mechanism is simple information retention in representations optimized for downstream tasks. CapRecover argues that intermediate visual features are not merely compressed codes but preserve task-relevant semantics, and that deeper convolutional layers encode significantly more semantic information than shallow layers (Xiu et al., 30 Jul 2025). FIA-Flow reaches a similar conclusion from the attack side: even in a black-box setting, aligned and refined latent inversion can reconstruct recognizable images from leaked split-DNN features, implying that these representations preserve fine-grained appearance, texture, spatial layout, and object identity (Ren et al., 19 Nov 2025). The 2018 intermediate-feature-compression study does not present a formal privacy analysis, but explicitly argues that intermediate features retain more spatial structure and generalize across multiple downstream tasks; this suggests why such features are useful and leak-prone at the same time (Chen et al., 2018).

Architecture can make leakage especially severe. PEEL attributes strong inference-time leakage in residual networks to skip connections, formalizing a preactivation residual block as

$z = \mathrm{norm}(p_\psi(h)) \in \mathbb{R}^{d}.$9

and interpreting the block output as a noisy version of the input. Because the skip path preserves a strong linear trace of $h$0, block outputs remain highly invertible (Arif et al., 8 Apr 2025). Split-U-Net observes a different depth effect for image inversion: earlier activation maps such as $h$1, $h$2, and $h$3 are most vulnerable because they preserve high spatial resolution, whereas deeper activations are less informative for reconstruction (Roth et al., 2022). Taken together, these results distinguish semantic leakage in deeper features from near-input reconstruction leakage in shallow features.

Objective design and bottleneck expressivity form another mechanism. In concept-based models, leakage rises when concept supervision is insufficient, when concept representations are more expressive than the underlying concept type, when the concept set is incomplete, or when the final head is misspecified (Parisini et al., 18 Apr 2025). CEMs are presented as especially prone to this effect because the supervised scalar concept probability can remain concept-accurate while the associated vector embeddings store arbitrary extra information (Parisini et al., 18 Apr 2025). The tree-based CBM study makes the same point in structural terms: soft concept probabilities can extend hard concept decision paths in precisely those regions where the hard concepts are incomplete (Ragkousis et al., 2024).

Leakage can also be induced by evaluation or sampling geometry rather than by representational capacity alone. In molFTP, fragment-target prevalence statistics are global supervised features, so if they are computed on the entire dataset before cross-validation, a held-out molecule can influence the feature weights assigned to its own fragments. The paper explicitly calls this feature leakage, distinct from simple train/test overlap (Godin, 7 Oct 2025). In the CA1 MaxEnt study, omitted higher-order, temporal, or hidden-state effects can be absorbed into fitted first-order parameters whenever they correlate with included sufficient statistics, with local leakage governed by

$h$4

That is, omitted structure can reappear as changes in fitted coefficients rather than as obvious residual error (Safaai et al., 1 Jun 2026).

A temporal version of the same problem appears in student analytics. Features such as cumulative GPA computed after the prediction point, total failed courses over the full record, or full-trajectory inactivity measures are leakage-prone because they encode post-cutoff information. CAPIRE treats this as a feature-design problem and uses VOT to enforce disjoint observation and outcome windows (Paz, 14 Nov 2025).

4. Empirical manifestations across application domains

The literature documents intermediate feature leakage in transfer learning, privacy attacks, concept-based interpretability, and leakage-aware data construction.

Setting	Intermediate representation	Reported leakage manifestation
Cross-hospital chest X-ray transfer	Frozen backbone features $h$5 and projection features $h$6	Site identity remains linearly decodable well above chance (Bouaziz et al., 31 Mar 2026)
Split DNN and VLM inference	Intermediate visual features	Labels, captions, and images can be recovered from features (Xiu et al., 30 Jul 2025, Ren et al., 19 Nov 2025)
Residual-network inference	Residual block outputs	Block-wise inputs and often the original image can be reconstructed (Arif et al., 8 Apr 2025)
Split learning for segmentation	Shared activation maps and gradients	MRI slices can be reconstructed; gradients may reveal labels (Roth et al., 2022)
Vertical federated learning	Forward cut-layer embeddings	Private labels can be inferred from embeddings (Sun et al., 2022)
Concept bottleneck models	Predicted concepts or concept embeddings	Bottlenecks encode task-relevant information beyond intended concepts (Parisini et al., 18 Apr 2025, Ragkousis et al., 2024)

In cross-hospital chest X-ray transfer, multi-site SSL improves RSNA AUC from $h$7 with ImageNet initialization to $h$8, but the representation remains heavily site-bearing. On backbone features $h$9, site-probe accuracy is $z$0 before SSL, $z$1 after SSL-only, and $z$2 after CanonicalF, with chance at $z$3; on projection features $z$4, the corresponding values are $z$5, $z$6, and $z$7 (Bouaziz et al., 31 Mar 2026). The empirical point is that transfer can improve substantially while leakage remains well above chance.

Semantic leakage in split models is likewise substantial. CapRecover reaches 83.35% Top-1 label accuracy on CIFAR-10 with ResNet50 and 92.71% with CLIP ViT-32; on TinyImageNet it still obtains 60.13% Top-1 with ResNet50 and 72.62% Top-1 with 91.60% Top-5 on CLIP ViT-32. For caption reconstruction on COCO2017, CLIP ViT-16 reaches BLEU-1 $z$8, ROUGE-L $z$9, and 84.38% cosine-similarity success (Xiu et al., 30 Jul 2025). FIA-Flow strengthens the same conclusion from the image-reconstruction side, reporting for ResNet-50 L1-2 PSNR 30.01, SSIM 0.814, LPIPS 0.100, and reconstruction classification accuracy 71.3, and for ResNet-50 L4-2 PSNR 20.31, SSIM 0.584, LPIPS 0.397, and accuracy 36.8 (Ren et al., 19 Nov 2025).

Residual architectures show particularly severe inversion. On CelebA, PEEL reports MSE $I(y;\hat{c}\mid c)=H(y\mid c)-H(y\mid \hat{c},c),$0 for class 1 and $I(y;\hat{c}\mid c)=H(y\mid c)-H(y\mid \hat{c},c),$1 for class 2, compared with KEDMI+LOMMA at $I(y;\hat{c}\mid c)=H(y\mid c)-H(y\mid \hat{c},c),$2 and $I(y;\hat{c}\mid c)=H(y\mid c)-H(y\mid \hat{c},c),$3 respectively, yielding the paper’s “roughly 13×–25× lower MSE” claim (Arif et al., 8 Apr 2025). Split-U-Net reports that inversion of $I(y;\hat{c}\mid c)=H(y\mid c)-H(y\mid \hat{c},c),$4 is “practically non-distinguishable” from the original augmented image, with leakage decreasing as depth increases (Roth et al., 2022).

Forward embeddings in vertical federated learning exhibit a distinct but related label channel. On Criteo, the spectral attack reaches leak AUC around 0.78 on the last two layers, including the cut layer; on Avazu, vanilla leak AUC is 0.7262 (Sun et al., 2022). In concept-based models, leakage manifests even when standard concept metrics look satisfactory. The information-theoretic CBM study reports that CTL and ICL strongly correlate with deterioration under interventions, that CTL is usually the stronger signal, and that soft or logit CBMs frequently exhibit substantial leakage even when concept accuracy is high (Parisini et al., 18 Apr 2025).

5. Mitigation, control, and leakage-aware design

Mitigation strategies fall into measurement-aware training, representation perturbation, architectural placement, bottleneck restriction, and evaluation-safe feature construction. In the chest X-ray study, adversarial site confusion on backbone features reduces measured leakage but does not reliably improve downstream transfer: CanonicalF lowers site-probe accuracy on $I(y;\hat{c}\mid c)=H(y\mid c)-H(y\mid \hat{c},c),$5 from $I(y;\hat{c}\mid c)=H(y\mid c)-H(y\mid \hat{c},c),$6 to $I(y;\hat{c}\mid c)=H(y\mid c)-H(y\mid \hat{c},c),$7, yet RSNA AUC changes from $I(y;\hat{c}\mid c)=H(y\mid c)-H(y\mid \hat{c},c),$8 for SSL-only Frozen to $I(y;\hat{c}\mid c)=H(y\mid c)-H(y\mid \hat{c},c),$9 for CanonicalF Frozen (Bouaziz et al., 31 Mar 2026). The lesson is methodological as much as algorithmic: leakage should be measured directly rather than inferred from transfer performance.

Several privacy defenses attempt to make leaked features unusable. CapRecover proposes reversible Gaussian noise injection at each layer,

$\hat{c}$0

with local noise removal before the next layer, so that the model’s final computation is unchanged while exposed intermediate representations become harder to invert. The paper reports inference-time overhead under 1% and large ROUGE-L drops for deeper layers under attack (Xiu et al., 30 Jul 2025). Split-U-Net proposes avoiding early activations, applying dropout with $\hat{c}$1, adding Gaussian differential privacy noise with $\hat{c}$2, and, if needed, homomorphic encryption or secure multi-party computation (Roth et al., 2022). FIA-Flow, by contrast, shows that common defenses such as Noise + NoPeek and DISCO reduce but do not eliminate split-DNN leakage (Ren et al., 19 Nov 2025).

In vertical federated learning, the defense is to minimize the distance correlation between forward embeddings and labels at the label party. The added objective

$\hat{c}$3

pushes embeddings toward label independence, and the paper reports that leak AUC can be reduced to about 0.5 while test AUC on Criteo remains around 0.78; on Avazu, leak AUC drops from 0.7262 to 0.5089 with $\hat{c}$4 while test AUC changes from 0.7532 to 0.7502 (Sun et al., 2022).

Concept-based models rely more on bottleneck discipline than on perturbation. Hard CBMs are reported to have vanishing CTL and ICL by construction, whereas CEMs exhibit substantial leakage regardless of hyperparameter choice (Parisini et al., 18 Apr 2025). Tree-based mixed CBMs use a leakage-free global hard tree and allow soft sub-tree extensions only inside selected leaves, thereby localizing leakage rather than permitting it throughout the model (Ragkousis et al., 2024). The 2025 information-theoretic measurement study argues that leakage quantification itself is estimator-sensitive and reports XGBoost as the most reliable conditional-entropy estimator among the tested options (Makonnen et al., 13 Apr 2025).

Leakage-aware feature engineering addresses a different layer of the problem. molFTP introduces dummy masking and key-LOO to prevent held-out molecules from influencing their own supervised fragment-prevalence features during cross-validation (Godin, 7 Oct 2025). CAPIRE enforces temporal honesty through VOT, forbidding post-hoc aggregates, final GPA, end-of-program completion statistics, label-derived features, and preprocessing fitted on full data before splitting (Paz, 14 Nov 2025). These cases show that leakage control may be built into data construction rather than only into model architecture.

6. Interpretation, identifiability, and recurrent misconceptions

A central misconception is that less measured leakage automatically implies better downstream behavior. The chest X-ray transfer study directly rejects this inference: multi-site SSL is the main driver of transfer gains, while adversarial site confusion is primarily a leakage-reduction tool and does not reliably improve RSNA AUC (Bouaziz et al., 31 Mar 2026). A related misconception in concept-based modeling is that high concept accuracy guarantees interpretability. Both information-theoretic CBM studies argue that the bottleneck may remain heavily contaminated even when concept prediction metrics are strong, because the learned concept representation can store extra task signal or interconcept dependencies (Parisini et al., 18 Apr 2025, Makonnen et al., 13 Apr 2025).

Another recurrent misconception is that predictive success identifies mechanism. The CA1 hippocampal MaxEnt study shows that a direct conditional maximum-entropy model can recover a large fraction of entropy and predict raw coactivities well under leakage-prone sampling while still failing to identify the true higher-order or temporal response rule. Approximately half of the selected leakage-enriched local tables that appear first-order under empirical weights become distribution-sensitive under balanced reweighting, with roughly $\hat{c}$5 retained and $\hat{c}$6 distribution-sensitive (Safaai et al., 1 Jun 2026). The paper’s interpretation is explicit: direct entropy-explained fractions and raw coactivity fits should be read as prediction under the observed state distribution, not as evidence that omitted mechanisms are absent.

Privacy-oriented work also repeatedly challenges the assumption that internal features are safe because raw inputs remain local. Split-U-Net states that split learning is not privacy-free just because raw medical images never leave the site (Roth et al., 2022). The vertical federated learning study shows that protecting only gradients is insufficient because forward embeddings can leak labels even under Marvell or Label DP (Sun et al., 2022). The intelligent-sensing compression paper presents intermediate features as more favorable for privacy than raw signals, but does not claim formal protection; this suggests that usefulness for multiple downstream tasks and privacy safety are not the same property (Chen et al., 2018).

Across these literatures, intermediate feature leakage is best understood as a representational mismatch between what an intermediate state is intended to encode and what it in fact makes recoverable. In some settings the mismatch is privacy-relevant, in others it is a threat to interpretability, transfer evaluation, or mechanistic inference. The shared methodological consequence is that internal representations must be audited as objects in their own right rather than treated as benign byproducts of successful prediction.