CECAS: Causal Framework for Visual Counterfactuals

• CECAS is a framework that produces counterfactual images by minimally perturbing inputs to flip classifier decisions while retaining non-causal, spurious features.
• It employs a two-stage architecture combining targeted adversarial steering with diffusion-based inpainting to balance validity, sparsity, proximity, and realism.
• Empirical results on datasets like CelebA and ImageNet subsets demonstrate high flip rates and improved preservation of spurious factors, underscoring its practical impact.

Searching arXiv for papers on CECAS and related counterfactual visual explanation work. CECAS denotes the framework introduced in “Counterfactual Visual Explanation via Causally-Guided Adversarial Steering” (Qiao et al., 14 Jul 2025). It is a method for generating visual counterfactual explanations: given an image $x$ and a classifier $f$, it seeks a modified image $x' = x + \delta$ whose prediction flips to a specified target label while maintaining validity, sparsity, proximity, and realism. Its defining claim is that counterfactuals should preferentially intervene on factors causally related to the label while preserving spurious factors, thereby reducing unintended alterations and semantically incoherent edits. The acronym is unrelated to CESAR, the CERN Experimental areas Software Renovation control system (Rae et al., 2022), and to CESIUM, the ComplEx System Integrated Utilities Model for complex engineered system development (Meluso et al., 2021).

1. Problem setting and conceptual motivation

In CECAS, a counterfactual explanation answers the local question of what minimal change to an input would make the model predict a different target label. For image $x \in \mathcal{X}$, label $y \in \mathcal{Y} := \{1,\dots,K\}$, and classifier $f:\mathcal{X}\to\mathcal{Y}$, the objective is to find a perturbation $\delta$ such that $x' := x + \delta$ changes the prediction to a target $y'$ (Qiao et al., 14 Jul 2025). The paper treats four properties as central: validity, proximality, sparsity, and realism.

The framework is motivated by three deficiencies attributed to earlier visual counterfactual methods. First, methods driven by global similarity penalties such as $\ell_1$ can still introduce local artifacts, including color shifts, shape distortions, extraneous texture changes, and unintended attribute changes. Second, existing approaches may exploit spurious correlations because they optimize whatever image cues most easily flip the classifier. Third, even diffusion-based or generative approaches, although stronger on realism, generally do not distinguish causally relevant factors from spurious factors or nuisance variation.

The paper uses concrete examples to characterize these failure modes. In autonomous driving, a method may change an irrelevant sign-shape cue rather than the causally meaningful content. In face aging, it may introduce beard-like dark patches because beard is spuriously correlated with old in the training data. CECAS is positioned as a response to these cases: it aims to generate counterfactuals that alter the decision through semantically meaningful directions rather than shortcut features (Qiao et al., 14 Jul 2025).

2. Causal abstraction and intervention criterion

CECAS does not construct a full structural causal model with explicit structural equations for all semantic concepts in the image. Instead, it adopts a coarse causal abstraction with variables $f$0, $f$1, $f$2, $f$3, $f$4, $f$5, and $f$6: causal factors $f$7 directly determine the label $f$8, both causal factors $f$9 and spurious factors $x' = x + \delta$0 contribute to the observed image $x' = x + \delta$1, and a semantically meaningful perturbation $x' = x + \delta$2 transforms $x' = x + \delta$3 into the counterfactual image $x' = x + \delta$4 (Qiao et al., 14 Jul 2025).

The graph structure emphasized in the paper includes $x' = x + \delta$5, $x' = x + \delta$6, and $x' = x + \delta$7, so that $x' = x + \delta$8 is a collider in the form $x' = x + \delta$9. The paper further stresses that conditioning on $x \in \mathcal{X}$0 can induce a spurious path $x \in \mathcal{X}$1, which formalizes why purely image-level optimization can exploit label-associated but noncausal cues. The key modeling assumption is that causal factors $x \in \mathcal{X}$2 and spurious factors $x \in \mathcal{X}$3 are independent, written as $x \in \mathcal{X}$4, and therefore can be separated into orthogonal subspaces in representation space.

Within this formulation, a causal intervention is approximated operationally rather than structurally. An ordinary perturbation is any change that flips the model. By contrast, a CECAS intervention is a change that drives the target prediction while leaving spurious mechanisms invariant. The paper makes this invariance concrete by extracting a spurious component from internal representations and penalizing changes to it during optimization. This does not amount to a full Pearl-style intervention semantics, but it provides an explicit criterion for distinguishing causally guided edits from unconstrained adversarial steering (Qiao et al., 14 Jul 2025).

3. Two-stage architecture and optimization pipeline

The method consists of two stages: initial causally-guided counterfactual generation, followed by causally-guided post refinement (Qiao et al., 14 Jul 2025). In the first stage, CECAS starts from targeted PGD-style optimization. Standard targeted adversarial steering updates the image toward a target class $x \in \mathcal{X}$5 through a projected gradient step, but the paper argues that this alone is insufficient because it may alter spurious factors.

To address this, the explained classifier is viewed as a feature extractor plus a linear classification head. A representation $x \in \mathcal{X}$6 is extracted, a spurious factor $x \in \mathcal{X}$7 is isolated by subspace projection under the assumption $x \in \mathcal{X}$8, and an auxiliary classifier $x \in \mathcal{X}$9 is trained on the spurious factor. The modified optimization then combines a target-label loss with a spurious-preservation loss:

$y \in \mathcal{Y} := \{1,\dots,K\}$0

where $y \in \mathcal{Y} := \{1,\dots,K\}$1 denotes the DDPM-plus-classifier joint pipeline, $y \in \mathcal{Y} := \{1,\dots,K\}$2 enforces target-label validity, $y \in \mathcal{Y} := \{1,\dots,K\}$3 penalizes changes in the spurious representation, and $y \in \mathcal{Y} := \{1,\dots,K\}$4 controls the trade-off between attack strength and causal constraint (Qiao et al., 14 Jul 2025).

DDPM regularization is used in this stage to suppress high-frequency adversarial noise. The paper describes the standard forward and reverse diffusion processes and conceptually treats denoising as a manifold prior that removes implausible noise while retaining semantically coherent content. The result of stage 1 is an initial counterfactual image that is both target-directed and constrained against altering extracted spurious factors.

Stage 2 addresses a different issue: concept-level preservation is too soft to guarantee pixel-level sparsity. CECAS therefore generates an auxiliary $y \in \mathcal{Y} := \{1,\dots,K\}$5-regularized target counterfactual,

$y \in \mathcal{Y} := \{1,\dots,K\}$6

not as the final output but as a localization device. The local normalized difference between $y \in \mathcal{Y} := \{1,\dots,K\}$7 and $y \in \mathcal{Y} := \{1,\dots,K\}$8 is thresholded to construct a binary mask, optionally enlarged by dilation, that identifies editable regions. Diffusion-based inpainting, following RePaint, then restores the rest of the image from the original while preserving the editable regions. The final output is thus produced by semantically steering first and spatially refining second (Qiao et al., 14 Jul 2025).

4. Experimental design, datasets, and evaluation protocol

The framework is evaluated on eight datasets or subsets spanning faces, traffic scenes, and ImageNet-derived animal-class transitions (Qiao et al., 14 Jul 2025).

Dataset group	Configuration
CelebA	$y \in \mathcal{Y} := \{1,\dots,K\}$9; Smile and Age
CelebA-HQ	$f:\mathcal{X}\to\mathcal{Y}$0; Smile and Age
BDD-OIA	traffic scenes; $f:\mathcal{X}\to\mathcal{Y}$1
ImageNet subsets	Cougar vs. Cheetah; Sorrel vs. Zebra; Egyptian Cat vs. Persian Cat

The classifier backbone is ResNet-50 pretrained by PyTorch for ImageNet, and DenseNet-121 with pretrained weights from prior baselines for the other datasets. The diffusion models use the same pretrained DDPM weights as the baselines for fairness. The comparison set includes STEEX, DiVE, DiVE$f:\mathcal{X}\to\mathcal{Y}$2, DiME, and ACE, although some experiments report only the strongest baselines because of space or unavailable reported numbers.

The evaluation protocol is organized around four counterfactual properties. Validity is measured by Flip Rate (FR) and Counterfactual Transition (COUT). Sparsity and proximity are measured by LPIPS, Correlation Difference (CD), and $f:\mathcal{X}\to\mathcal{Y}$3, where $f:\mathcal{X}\to\mathcal{Y}$4 is SimSiam cosine similarity. Realism is measured by FID and sFID, with sFID introduced because standard FID can underestimate changes when original and counterfactual images share many pixels. The paper also adds a VLM-based evaluation using GPT-4o with custom prompts to score validity, sparsity, proximity, and realism from 1 to 10, then average them. Reported hyperparameters vary by dataset, including $f:\mathcal{X}\to\mathcal{Y}$5, $f:\mathcal{X}\to\mathcal{Y}$6, $f:\mathcal{X}\to\mathcal{Y}$7, PGD iterations, and diffusion steps; for example, CelebA Smile uses $f:\mathcal{X}\to\mathcal{Y}$8, $f:\mathcal{X}\to\mathcal{Y}$9, $\delta$0, 50 PGD iterations, and 500 diffusion steps, whereas BDD-OIA uses $\delta$1, $\delta$2, $\delta$3, 50 PGD iterations, and 1000 diffusion steps (Qiao et al., 14 Jul 2025).

5. Empirical performance and qualitative behavior

The reported results present CECAS as strongest overall on multiple datasets, while not being best on every individual metric (Qiao et al., 14 Jul 2025). On CelebA Smile, it achieves FR $\delta$4, COUT $\delta$5, LPIPS $\delta$6, $\delta$7 $\delta$8, and FID $\delta$9, all listed as best; its sFID $x' := x + \delta$0 is second-best, slightly behind DiME’s $x' := x + \delta$1, and its CD $x' := x + \delta$2 is slightly worse than ACE $x' := x + \delta$3 and DiME $x' := x + \delta$4. On CelebA Age, it records FR $x' := x + \delta$5, COUT $x' := x + \delta$6, LPIPS $x' := x + \delta$7, CD $x' := x + \delta$8, and sFID $x' := x + \delta$9 as best, while FID $y'$0 is slightly behind ACE’s $y'$1 and $y'$2 $y'$3 is slightly behind ACE’s $y'$4.

On BDD-OIA, the method ties ACE on FR at $y'$5, and reports the best COUT at $y'$6 and the best LPIPS at $y'$7, with $y'$8 $y'$9 very close to ACE’s $\ell_1$0. Its FID and sFID are slightly worse than ACE, at $\ell_1$1 versus $\ell_1$2 and $\ell_1$3 versus $\ell_1$4, respectively. On the ImageNet subsets, CECAS generally improves validity and transition strength relative to ACE. For Sorrel $\ell_1$5 Zebra, FR rises to $\ell_1$6 from $\ell_1$7, COUT to $\ell_1$8 from $\ell_1$9, and both FID and sFID improve, although LPIPS is slightly worse. Similar patterns hold for Cougar $f$00 Cheetah and Persian Cat $f$01 Egyptian Cat: validity and realism-style metrics generally improve, but some similarity metrics do not.

The qualitative findings are central to the paper’s interpretation. On facial datasets, CECAS is reported to avoid inexplicable color shifts, shape distortions, exaggerated facial creases, and dark chin patches in female faces during young-to-old transitions. That beard-like artifact is treated as the clearest example of a spurious correlation that other methods may exploit. On BDD-OIA and the animal subsets, the method produces more localized, human-perceptible modifications while preserving more of the rest of the image. The GPT-4o-based evaluation reports CECAS as best on seven of eight datasets, including CelebA Smile, CelebA Age, CelebA-HQ Smile, CelebA-HQ Age, Sorrel-Zebra, Cougar-Cheetah, and Persian Cat-Egyptian Cat; BDD-OIA is the only case where ACE slightly exceeds it, $f$02 versus $f$03 (Qiao et al., 14 Jul 2025).

6. Assumptions, limitations, and terminological boundaries

The paper is explicit that CECAS is not a full causal solution (Qiao et al., 14 Jul 2025). Its causal formulation models only a coarse separation between causal factors $f$04 and spurious factors $f$05, and it does not explicitly represent finer causal relations among multiple semantic variables. Its effectiveness depends on the quality of spurious-factor extraction from representation space; if disentanglement is poor, the causal guidance may be noisy or incomplete. The assumption $f$06, which enables orthogonal projection into causal and spurious subspaces, is strong and may not hold exactly in real datasets.

The method is also essentially white-box: it uses gradients and model internals. Relative to ordinary counterfactual generation, it adds spurious-factor extraction, auxiliary classifier training, DDPM denoising, mask generation, and diffusion inpainting, all of which increase computational cost and implementation complexity. The trade-off hyperparameter $f$07 is especially important. The paper reports that as $f$08 increases, realism initially improves because spurious changes are suppressed, but overly large $f$09 can reduce flip rate and eventually harm realism because necessary modifications become overconstrained. The authors further note that automatic balancing among validity, sparsity, proximity, and realism remains future work.

A common source of confusion is purely terminological. In arXiv records, “CECAS” refers to the counterfactual-visual-explanation framework described above (Qiao et al., 14 Jul 2025). The near-homophonous CESAR concerns CERN Experimental Area beam control software (Rae et al., 2022), and CESIUM refers to a simulation framework for complex engineered system development (Meluso et al., 2021). Within machine learning and explainable AI, however, CECAS specifically denotes a two-stage framework that combines targeted adversarial steering, spurious-factor preservation, diffusion denoising, and mask-based inpainting to produce counterfactual images with a balanced trade-off among validity, sparsity, proximity, and realism (Qiao et al., 14 Jul 2025).