Machine Unlearning in TSEP

• Machine Unlearning TSEP is a framework that integrates deletion into neural propensity-score models, ensuring causal effect estimation reflects retained data only.
• It employs methodologies such as pair-wise and random removal, with RMSE variations illustrating trade-offs in maintaining accurate treatment effect predictions.
• Complementary strategies like retain-free editing, spectral reweighting, and architectural partitioning enhance unlearning efficiency while safeguarding data privacy.

Searching arXiv for papers relevant to machine unlearning and TSEP-related settings. First, I’ll look up the direct causal-inference/TSEP-related paper, then machine unlearning methods that explicitly discuss relevance to TSEP-like pipelines. Searching arXiv for “machine unlearning causal inference propensity score TSEP”. Machine Unlearning TSEP denotes the incorporation of machine unlearning into a Treatment-Specific Effect Prediction or treatment effect estimation pipeline. In the literature surveyed here, the most direct formulation appears in neural propensity-score modeling for causal inference, where the objective is to remove the influence of designated individuals or matched treated–control pairs from the propensity model and then propagate updated scores to downstream matching, weighting, or related TSEP components. Surrounding machine unlearning research does not usually present TSEP as a standalone unlearning algorithm, but it supplies several mechanisms—retain-free editing, sensitivity suppression, spectral reweighting, architectural partitioning, and explicit test-time deletion—that are described as relevant or complementary to TSEP-like systems (Ramachandra et al., 2023, Chen et al., 12 Nov 2025).

1. Conceptual placement

Within the broader machine unlearning taxonomy, the relevant distinction is between exact unlearning, approximate unlearning, strong and weak guarantees, and between data reorganization and model manipulation. The survey literature formalizes the basic setting with a training dataset $\mathcal{D}$, an unlearning subset $\mathcal{D}_u$, a remaining set $\mathcal{D}_r = \mathcal{D}\setminus \mathcal{D}_u$, a retrained reference model $$\mathbf{w}_r = \mathcal{A}(\mathcal{D}\setminus\mathcal{D}_u)$$, and an unlearned model $\mathbf{w}_u$ that should behave as though $\mathcal{D}_u$ had never been used. Exact unlearning requires distributional equivalence to retraining, whereas approximate or certified variants allow bounded divergence; utility, consistency, and verifiability are treated as core desiderata (Xu et al., 2023).

In the TSEP-specific literature, the causal-inference formulation is the clearest anchor. The key object is a propensity-score model $e(X)=P(T=1\mid X)$, learned by a neural network and used for matching or weighting in downstream treatment effect estimation. Unlearning is then defined operationally as removing the influence of selected individuals or matched pairs by retraining or fine-tuning on the retain set, with the updated propensity scores subsequently serving the causal pipeline. Several later machine unlearning papers state that TSEP is not mentioned explicitly in their own text, but they identify their mechanisms as applicable to “TSEP-like” systems when selective deletion, privacy, or data access constraints arise (Ramachandra et al., 2023).

2. Direct formulation in causal inference and propensity-score pipelines

The direct TSEP-relevant formulation uses a neural propensity-score model trained on a causal dataset such as Lalonde. In the standard potential-outcomes setup, covariates are $X_i$, treatment is $T_i\in\{0,1\}$, and the propensity score is $e(X)=P(T=1\mid X)$. The paper focuses on unlearning at this propensity-score stage rather than at the downstream ATE, ATT, or ITE estimator. Its operational procedure is to train an initial propensity model on the full dataset, define a forget set, retrain from the original weights using only the retain set, and then examine how much the updated model’s propensity-score distribution and predictive behavior differ from the original model (Ramachandra et al., 2023).

Two forget-set constructions are studied. The first is propensity matching pair-wise removal, in which treated units are matched to nearest-neighbor controls in propensity space using $\mathcal{D}_u$0, and matched pairs are removed until roughly $\mathcal{D}_u$1 of the total data are covered. The second is random removal, with $\mathcal{D}_u$2 of treated and $\mathcal{D}_u$3 of controls selected without regard to propensity structure. The paper reports that Model 1, trained on the full dataset, has $\mathcal{D}_u$4; Model 2, fine-tuned on the retain set after pair-wise removal, has $\mathcal{D}_u$5; and Model 3, fine-tuned after random removal, has $\mathcal{D}_u$6. Distribution and histogram analyses show that pair-wise forgetting yields less overlap in the forget-set propensity distributions and a larger departure from the original model, while random removal leaves the distributions almost completely overlapping and produces only weak unlearning (Ramachandra et al., 2023).

The causal significance is specific. Pair-wise forgetting is described as aligned with causal structure because it removes matched treated–control units that are most similar under the assignment mechanism. At the same time, explicit numerical ATT or ATE values are not reported. The paper therefore frames TSEP relevance through the updated propensity module: once unlearning has altered $\mathcal{D}_u$7, those scores can be fed back into matching, inverse probability weighting, doubly robust estimators, or broader Treatment-Specific Effect Prediction pipelines (Ramachandra et al., 2023).

3. Mechanism families that can be imported into TSEP-like systems

Several later machine unlearning methods do not target causal inference directly, but they expose mechanism-level patterns that are useful for TSEP-like deployments. One example is Probing then Editing, a retain-free framework that uses only the original model and the to-be-forgotten data. It probes the decision-boundary neighborhood of the forget class by projected gradient ascent, generates self-consistent editing instructions from the model’s own predictions, then alternates a push branch that dismantles the forget-class region with a pull branch that performs masked knowledge distillation on non-forget classes. The paper explicitly states that, although TSEP is not mentioned, principles such as boundary-centric forgetting, pseudo-label editing targets, and mask-based distillation are relevant to TSEP-like frameworks (Chen et al., 12 Nov 2025).

A second family focuses on stability allocation. Statistical-Roughness Adaptive Gradient Unlearning extends Adaptive Gradient Unlearning by computing layer-wise heavy-tailed spectral diagnostics, converting a fitted exponent $\mathcal{D}_u$8 into a bounded spectral stability weight $\mathcal{D}_u$9, and reweighting forget-set sensitivities so that spectrally stable layers absorb more of the unlearning update. The paper again states that TSEP is not mentioned explicitly, but that any TSEP-style method could in principle combine its own forget signal with this layer-wise stability prior. The practical implication is not a new causal estimand; it is a reallocation rule for where forgetting updates should be applied in a deep architecture (Partohaghighi et al., 10 Feb 2026).

A third family addresses coupled retain and forget structure. Under retain–forget entanglement, the retain set is split into an adjacent subset $\mathcal{D}_r = \mathcal{D}\setminus \mathcal{D}_u$0 and a remote subset $\mathcal{D}_r = \mathcal{D}\setminus \mathcal{D}_u$1. The method first uses an augmented Lagrangian to increase forget-set loss while keeping remote-retain loss fixed, then applies a gradient-projection step regularized by the Wasserstein-2 distance so that adjacent-retain performance can recover without compromising the forget objective. This is directly relevant when deleted and retained cohorts share semantically similar or highly correlated structure. In a TSEP setting, matched treated–control individuals or closely related covariate strata would plausibly create precisely this kind of entanglement; that implication is conceptual rather than directly tested in the causal paper (Cheng et al., 27 Mar 2026).

A fourth family removes dependence on retain data altogether. MU-Mis suppresses the contribution of forgotten samples by minimizing input sensitivity, using only the pretrained model and the forget set. Its core loss is

$\mathcal{D}_r = \mathcal{D}\setminus \mathcal{D}_u$2

where $\mathcal{D}_r = \mathcal{D}\setminus \mathcal{D}_u$3 is the true class and $\mathcal{D}_r = \mathcal{D}\setminus \mathcal{D}_u$4 is a randomly chosen irrelevant class. The method is explicitly described as remaining-data-free and is positioned as complementary to TSEP-like projection methods because it works through input Jacobian structure rather than explicit retain-set projections (Cheng et al., 2024).

4. Architectural and systems patterns for deletion-aware TSEP

A separate line of work moves unlearning from parameter editing to system design. Dataset Condensation Driven Machine Unlearning introduces a retain dataset reduction framework that clusters the original data, condenses each cluster into a representative image, constructs a reduced retain dataset $\mathcal{D}_r = \mathcal{D}\setminus \mathcal{D}_u$5, and then applies modular unlearning to beginning, intermediate, and final network components. It also proposes an unlearning metric based on gradient orthogonality between $\mathcal{D}_r = \mathcal{D}\setminus \mathcal{D}_u$6 and $\mathcal{D}_r = \mathcal{D}\setminus \mathcal{D}_u$7. Although developed for image classification, the framework is explicitly described as complementary to parameter-editing methods such as TSEP because it reduces the cost of the retain-side retraining or fine-tuning stage (Khan, 2024).

Sharded architectures supply another route. In paralinguistic speech processing, SISA++ partitions training data into shards, retrains only affected shard models after user-level deletion requests, and then merges models by weight averaging. The paper studies Speech Emotion Recognition and Depression Detection, but the architectural point is broader: user-specific or cohort-specific forget requests can be localized to shards rather than handled by global retraining. That pattern is directly compatible with TSEP pipelines that already separate data by institution, cohort, or treatment regime (Phukan et al., 2 Jun 2025).

The most radical architectural response is to design models that are easy to forget by construction. Deep semi-parametric models keep a parametric backbone but make predictions depend explicitly on a support set $\mathcal{D}_r = \mathcal{D}\setminus \mathcal{D}_u$8 supplied at test time: $\mathcal{D}_r = \mathcal{D}\setminus \mathcal{D}_u$9 Unlearning is implemented by explicit test-time deletion,

$$\mathbf{w}_r = \mathcal{A}(\mathcal{D}\setminus\mathcal{D}_u)$$0

with no parameter update. The paper reports that on ImageNet classification these models reduce the prediction gap relative to a retrained baseline by $$\mathbf{w}_r = \mathcal{A}(\mathcal{D}\setminus\mathcal{D}_u)$$1 and achieve over $$\mathbf{w}_r = \mathcal{A}(\mathcal{D}\setminus\mathcal{D}_u)$$2 faster unlearning than existing parametric approaches. For TSEP-like systems with repeated deletion requests, this suggests an alternative to post-hoc editing: store part of the relevant training evidence in an editable memory rather than only in weights (Zheng et al., 24 Mar 2026).

5. Evaluation, auditing, and the problem of fake forgetting

The causal-inference paper evaluates unlearning mainly through the propensity-score model itself: RMSE on the original dataset, plus kernel density and histogram analyses of propensity-score distributions before and after forgetting. That is sufficient to show that pair-wise removal perturbs the assignment mechanism more strongly than random deletion, but it does not provide explicit ATE or ATT estimates, and it does not audit whether the model still regards deleted labels or deleted membership status as plausible at an uncertainty-aware level (Ramachandra et al., 2023).

Later unlearning work introduces richer diagnostics that are directly relevant to TSEP evaluation. Conformal-prediction-based evaluation argues that low forget accuracy or favorable MIA scores can still mask “fake unlearning” when the ground-truth label remains in the model’s conformal prediction set. It proposes the Conformal Ratio,

$$\mathbf{w}_r = \mathcal{A}(\mathcal{D}\setminus\mathcal{D}_u)$$3

and the MIA Conformal Ratio,

$$\mathbf{w}_r = \mathcal{A}(\mathcal{D}\setminus\mathcal{D}_u)$$4

to distinguish genuine exclusion of the true label or member status from mere top-1 misclassification. This suggests that a TSEP pipeline should not be evaluated only by propensity-score RMSE or downstream treatment-effect stability; uncertainty-aware exclusion metrics may be needed when deletion requests have legal or clinical significance (Shi et al., 31 Jan 2025).

Behavioral-alignment metrics provide another audit layer. SRAGU evaluates unlearning by closeness to a gold retrained reference model using empirical prediction-divergence

$$\mathbf{w}_r = \mathcal{A}(\mathcal{D}\setminus\mathcal{D}_u)$$5

and average KL divergence on the forget set, together with membership-inference auditing. RUM introduces Tug-of-War,

$$\mathbf{w}_r = \mathcal{A}(\mathcal{D}\setminus\mathcal{D}_u)$$6

to expose how forget-set structure changes difficulty. A plausible implication is that TSEP unlearning should be audited against a retrained reference on both deleted and retained cohorts, rather than only by raw predictive fit (Partohaghighi et al., 10 Feb 2026, Zhao et al., 2024).

6. Difficulty factors, limitations, and future directions

The present TSEP-relevant literature remains narrow in direct causal scope. The explicit causal paper is limited to a neural propensity-score model on the Lalonde dataset, and it reports RMSE and distributional diagnostics rather than direct numerical ATE or ATT changes. Its unlearning procedure is also approximate rather than certified. More generally, most of the broader mechanisms discussed above—retain-free probing and editing, sensitivity suppression, spectral weighting, entanglement-aware optimization, stochastic teacher distillation, and condensation-driven modular unlearning—do not offer exact equivalence to retraining (Ramachandra et al., 2023, Chen et al., 12 Nov 2025).

At the same time, the difficulty analyses in the unlearning literature are highly relevant. RUM shows that forget-set difficulty depends on memorization level and embedding-space entanglement, and that different algorithms dominate in different regimes. The retain–forget entanglement paper shows that closely related retained samples can be disproportionately damaged when forgetting is performed as a single undifferentiated operation. These results suggest that TSEP unlearning should treat deleted cohorts as heterogeneous: matched pairs, high-memorization outliers, and semantically entangled retained subpopulations are unlikely to be well served by a single global deletion heuristic (Zhao et al., 2024, Cheng et al., 27 Mar 2026).

A deeper theoretical direction comes from overparameterized unlearning. In that setting, the paper argues that defining the unlearning solution as any loss minimizer on the retained set is inadequate, because the original interpolating model may already satisfy that condition. It instead proposes the minimum-complexity interpolator over the retained data and an orthogonal-gradient perturbation framework. For TSEP-like systems, this suggests a possible future foundation for defining what the correct post-deletion propensity or treatment-effect model should be when many interpolating solutions exist (Block et al., 28 May 2025).

The immediate research agenda is therefore threefold. First, direct TSEP work needs to move beyond propensity-score RMSE toward end-to-end causal estimands, heterogeneous treatment effects, and uncertainty-aware deletion audits. Second, deployment-oriented work needs architectures and data layouts that make deletion cheap under siloed or regulated data access. Third, theoretical work needs stronger guarantees for approximate unlearning in overparameterized, deep, and highly entangled settings. Across all three directions, the central lesson is stable: in TSEP, forgetting is not merely a request to lower accuracy on a deleted subset, but a requirement to remove that subset’s influence from the causal adjustment machinery while preserving valid behavior on the retained population.