Counterfactual Explanation in ML

• Counterfactual Explanation (CE) is a post-hoc explanation method that identifies minimally changed input alternatives to flip a model's decision, emphasizing contrastive reasoning.
• The framework encompasses diverse methodologies—optimization-based, rule-based, latent space, and uncertainty-aware—with trade-offs in feasibility, accuracy, sparsity, and runtime.
• Recent advances extend CE from local point explanations to global, regional, and distributional forms, addressing robustness, causality, and actionable recourse in decision systems.

Searching arXiv for the cited CE papers to ground the article and confirm current metadata. Counterfactual explanation (CE) is a post-hoc explanation paradigm that identifies alternative inputs that would lead a model to produce a different, typically desired, output. In the standard formulation, a CE is a minimally changed alternative input; in the broader literature, the object of explanation has expanded from a single point to rules, regions, distributions, ordered intervention plans, and even modified optimization or planning problems (Freiesleben, 2020). Across these formulations, CE remains contrastive: it explains a decision by specifying how the input, or a structured representation of it, would need to differ for the outcome to change (Bewley et al., 2024).

1. Formal foundations

A canonical mathematical template for CE seeks an alternative $x'$ near the factual input $x$ such that the model output moves to a desired target: $$\underset{x'\in X}{\arg\min}\; d(x,x') +\lambda\; d'(f(x'),y_{des}).$$ This shared optimization template underlies both much of the CE literature and much of the adversarial-example literature, but the two are not identical (Freiesleben, 2020). In the formal account that distinguishes them, a counterfactual is a minimally distant alternative $c_x$ such that $f(c_x)\neq f(x)$, whereas an adversarial is an $\epsilon$-alternative that is also misclassified relative to the true label. The decisive distinctions are therefore the relation to the true label and the proximity criterion: CEs require minimality, while adversarials require only bounded closeness (Freiesleben, 2020).

The local recourse formulation in tabular CE is made explicit by T-CREx. Given $f:\mathcal X\to\mathcal Y$, a reference point $x^0$, and a target set $Y^*\subseteq \mathcal Y\setminus\{f(x^0)\}$, standard point counterfactual explanation is

$$\text{CPE}(x^0\ |\ Y^*, P_\mathcal{X})=\underset{x^i\in \mathcal{X}:f(x^i)\in{Y^*}}{\arg\min\ \text{cost}(x^0, x^i\ |\ P_\mathcal{X})}.$$

This definition already accommodates feasibility and plausibility through dependence on the realistic input distribution $x$0 (Bewley et al., 2024).

Regression complicates this picture. One line of work still defines CE by a target equation such as $x$1, with generation performed for a pre-trained regressor and a predefined target output (Zhao et al., 2023). Another line argues that in regression the desired target value is often exogenous and arbitrary, and that the practical interpretation of geometric distance is likewise unclear; profit-based counterfactual explanation therefore replaces target-reaching with direct optimization of profit and interprets the distance term as the cost of modifying product attributes (Kinjo et al., 2 Jul 2026). This tension between target-matching and objective-driven formulations is one of the defining issues in contemporary CE for continuous outcomes.

2. Optimization criteria and recurring trade-offs

CE research is organized around a stable set of desiderata, but different methods instantiate them differently. In the T-CREx evaluation, the six desiderata are feasibility, accuracy, sparsity, complexity, consistency, and runtime; feasibility and accuracy are defined with respect to the realistic input distribution, sparsity counts changed features, complexity counts the number of finite rule terms, consistency measures the fraction of unique rules returned across the test set, and runtime measures computational cost (Bewley et al., 2024). Other papers emphasize plausibility, actionability, diversity, or robustness to perturbations and model updates (Černý et al., 10 Jul 2025).

These criteria are often in tension. T-CREx reports a clear trade-off mediated by the rule-accuracy threshold $x$2: larger $x$3 gives more accurate but less feasible, less sparse, and more complex rules (Bewley et al., 2024). In recommender systems, plausible CEs are defined by likelihood under a Sum-Product Network, and improving plausibility typically improves log-likelihood but worsens proximity in $x$4 distance (Černý et al., 10 Jul 2025). PBCE reframes the standard proximity penalty as economically meaningful adjustment cost,

$x$5

so that deviations are penalized not because counterfactuals should be geometrically close, but because product changes are costly (Kinjo et al., 2 Jul 2026).

A separate but related issue is actionable minimality. COLA argues that many already-valid CEs contain unnecessary feature changes and proposes post-hoc refinement using a joint distribution between factual and counterfactual samples, estimated by optimal transport, together with $x$6-SHAP, a joint-probability-informed Shapley attribution. In that view, minimality is not only a generation-time regularizer but also a refinement problem: one can prune unnecessary edits after CE generation while trying to preserve counterfactual effect (You et al., 2024). This suggests that “minimal” may refer to different objects—geometric distance, feature count, economic cost, or attributional necessity—depending on the application.

3. Methodological families

A major strand of CE research remains optimization-based. Ordered Counterfactual Explanation (OrdCE) extends standard actionable recourse from an unordered perturbation vector $x$7 to an ordered pair $x$8, where $x$9 is an execution order over changed features. The method introduces an order-dependent cost

$$\underset{x'\in X}{\arg\min}\; d(x,x') +\lambda\; d'(f(x'),y_{des}).$$0

and solves the resulting problem exactly by mixed-integer linear optimization for additive classifiers including linear models, tree ensembles, and two-layer ReLU MLPs (Kanamori et al., 2020). This makes feature interaction, including causal or asymmetric dependence, part of the CE object itself.

A second family is rule-based and surrogate-based. T-CREx replaces point counterfactuals by counterfactual rules $$\underset{x'\in X}{\arg\min}\; d(x,x') +\lambda\; d'(f(x'),y_{des}).$$1 and introduces metarules $$\underset{x'\in X}{\arg\min}\; d(x,x') +\lambda\; d'(f(x'),y_{des}).$$2 capturing regions where a given rule is optimal. Candidate rules are obtained from tree or random-forest surrogates; valid rules satisfy feasibility and accuracy thresholds; and every grid cell induced by maximal-valid-rule boundaries becomes a valid metarule under the chosen sparsity-plus-feasibility cost (Bewley et al., 2024). GLOBE-CE pursues a different global route: it defines a global CE as a shared translation direction $$\underset{x'\in X}{\arg\min}\; d(x,x') +\lambda\; d'(f(x'),y_{des}).$$3 together with instance-specific magnitudes $$\underset{x'\in X}{\arg\min}\; d(x,x') +\lambda\; d'(f(x'),y_{des}).$$4, so that $$\underset{x'\in X}{\arg\min}\; d(x,x') +\lambda\; d'(f(x'),y_{des}).$$5. This addresses the coverage–cost trade-off that arises when a single fixed transformation is imposed on an entire group (Ley et al., 2023).

A third family works in learned latent spaces. One method shapes the latent space of an autoencoder into a class-conditional Gaussian mixture and generates CEs by linear interpolation from the query embedding toward the target-class centroid while keeping a label-irrelevant latent code fixed (Zhao et al., 2023). A related regression method disentangles label-relevant from label-irrelevant information by learning a latent representation $$\underset{x'\in X}{\arg\min}\; d(x,x') +\lambda\; d'(f(x'),y_{des}).$$6 that is adversarially stripped of target information, then decoding $$\underset{x'\in X}{\arg\min}\; d(x,x') +\lambda\; d'(f(x'),y_{des}).$$7 together with a desired continuous output (Zhao et al., 2023). These methods emphasize manifold adherence and preservation of query-specific characteristics rather than explicit sparsity in the original input space.

A fourth family is uncertainty-aware. A white-box gradient-based approach uses predictive uncertainty of a classifier ensemble as an implicit proxy for realism and ambiguity, arguing that minimizing target-class cross-entropy also minimizes predictive entropy, hence steering CEs toward regions of low epistemic and aleatoric uncertainty without an auxiliary generative model (Schut et al., 2021). More recent Bayesian work makes uncertainty constraints explicit through $$\underset{x'\in X}{\arg\min}\; d(x,x') +\lambda\; d'(f(x'),y_{des}).$$8-safety and $$\underset{x'\in X}{\arg\min}\; d(x,x') +\lambda\; d'(f(x'),y_{des}).$$9-robustness, requiring high posterior predictive confidence and low predictive variance for the target class (Duell et al., 23 Jan 2026).

4. From local points to global, regional, and distributional explanations

The CE literature no longer treats the local point explanation as its only object. T-CREx makes this shift explicit by defining counterfactual rule explanation (CRE), where the local CE for $c_x$0 is the optimal rule among maximal-valid rules, and by defining metarules as regions of optimality: interpretable subsets of input space such that every point in the region receives the same optimal counterfactual rule (Bewley et al., 2024). The result is a population-level map of recourse structure rather than a bag of isolated individual explanations.

GLOBE-CE reaches a similar destination from a geometric angle. It argues that a global explanation should be a shared direction of change rather than a fixed translation or a post-hoc aggregation of many local CEs. For one-hot categorical variables it proves that any translation can be expressed as If/Then rules with one unique Then condition, and that scaled translations induce cumulative rule systems, which the paper terms a Cumulative Rules Chart (Ley et al., 2023). The global object is therefore neither a single point nor a static rule, but a shared directional structure with variable per-instance magnitude.

Regional robustness yields another enlargement of the CE object. In robust CE via robust optimization, the standard formulation

$c_x$1

is replaced by

$c_x$2

where $c_x$3 is an uncertainty set. The solution is a center point together with a whole region $c_x$4 of valid CEs, which is especially interpretable when $c_x$5 is an $c_x$6-box (Maragno et al., 2023).

Distributional Counterfactual Explanation (DCE) generalizes even further. DISCOUNT treats the factual object as an empirical input distribution $c_x$7, the target as an output distribution $c_x$8, and seeks a counterfactual input distribution $c_x$9 that stays close to $f(c_x)\neq f(x)$0 in sliced Wasserstein distance while driving $f(c_x)\neq f(x)$1 close to $f(c_x)\neq f(x)$2 in Wasserstein distance, with statistical confidence guaranteed through chance constraints and upper confidence limits (You et al., 2024). This replaces pointwise recourse by quantile-level and transport-level reasoning.

5. Robustness, causality, and the status of validity

Validity in CE is not a unitary concept. One notion is robustness to implementation error in the recourse itself: robust CE regions require that the recommended action remain valid under all perturbations in an uncertainty set, and the paper proves finite termination of an adversarial cutting-set algorithm for linear models, trees, random forests, gradient boosting machines, and ReLU neural networks under suitable Lipschitz conditions (Maragno et al., 2023). A second notion is robustness to model change. PSCE defines the $f(c_x)\neq f(x)$3-set

$f(c_x)\neq f(x)$4

and derives bounds on how predictive confidence and variance degrade when the posterior changes by a small KL divergence (Duell et al., 23 Jan 2026). VeriTraCER addresses the same problem from a training-and-verification perspective, defining a multiplicity set of nearby models that preserve the original prediction and certifying that the CE remains valid for all such local updates (Meyer et al., 2024).

A separate issue is causal validity. A recurrent misconception is that a CE generated from a predictive model is automatically a valid intervention in the world. Several papers reject this. Freiesleben emphasizes that CE, as such, is agnostic with respect to the true label, unlike adversarial examples (Freiesleben, 2020). The Pearl-based evaluation study goes further: when DiCE-generated CEs were checked against SCM-based counterfactuals on chain, fork, and collider structures, 10 out of 30 evaluations conflicted with Pearl’s method, with collider structures being especially problematic (Smith, 2023). PBCE likewise states explicitly that its recommendations are derived from predictive associations and do not guarantee causal relationships (Kinjo et al., 2 Jul 2026). The literature therefore separates model-relative contrastive validity from intervention-valid causal claims.

This also clarifies the distinction between CE and recourse. CE may be used to raise understanding, give guidance for future actions, or allow people to contest decisions, but a model-level CE need not automatically translate into real-world recourse (Freiesleben, 2020). Robustness to action perturbations, robustness to model updates, and causal reliability are now treated as distinct axes rather than as consequences of ordinary validity.

6. Domain-specific extensions and scope expansion

Recent work has pushed CE far beyond binary tabular classification. In management and marketing, PBCE formulates CE as

$f(c_x)\neq f(x)$5

so that the target output is endogenous to profit maximization rather than specified exogenously (Kinjo et al., 2 Jul 2026). In recommender systems, plausible CE is adapted to ranking: mixed-integer optimization is used to make a target item drop below a desired rank while keeping the counterfactual likely under the training distribution, as measured by a Sum-Product Network (Černý et al., 10 Jul 2025).

Other papers change the very object to which CE is applied. In automated planning, counterfactual scenarios seek minimal modifications to a planning problem $f(c_x)\neq f(x)$6, rather than to one plan, so that either some plan or all valid plans satisfy an $f(c_x)\neq f(x)$7 property $f(c_x)\neq f(x)$8 (Gigante et al., 29 Aug 2025). In integer optimization, weak and strong CE are defined over mutable model parameters $f(c_x)\neq f(x)$9, with weak CE requiring at least one optimal solution in a favored set $\epsilon$0 and strong CE requiring that every optimal solution lie in $\epsilon$1; the general decision problem is shown to be $\epsilon$2-complete even for binary integer programs with a single mutable constraint (Engelhardt et al., 20 Oct 2025). In multivariate time series forecasting with exogenous variables, CET-X optimizes over recent exogenous trajectories $\epsilon$3 to make the predicted target path track a desired trajectory $\epsilon$4, and evaluates the resulting CE by validity ($\epsilon$5-loss), proximity ($\epsilon$6-loss), total loss, temporal smoothness, and, when available, MAE against a theoretical solution (Kinjo, 10 Nov 2025).

These domain-specific developments reinforce a broader shift. CE is no longer confined to “find a nearby point that flips a classifier.” It now includes rule summaries, regions of validity, distributions, ordered intervention plans, Bayesian safety sets, optimization-parameter perturbations, planning-problem modifications, and sequential interventions under exogenous dynamics. This suggests that CE has become a general framework for contrastive model-relative reasoning, with the exact explanatory object determined by the structure of the decision system being studied.