Variation-Based Causal Discovery (VCEI)

• Variation-Based Causal Discovery (VCEI) is a method that leverages artificial sample reweighting to induce variation and reveal causal asymmetry based on the Independence of Cause and Mechanism.
• It employs kernel maximum mean discrepancy and a semi-definite relaxation to quantify changes in conditional stability, guiding reliable bivariate causal direction inference.
• The approach is applicable to diverse data types while requiring assumptions like acyclicity, causal sufficiency, and an existing causal link, highlighting both its versatility and limitations.

Variation-based causal discovery designates causal inference strategies that exploit changes, heterogeneity, or induced variation to reveal asymmetries that are not available from a single stationary distribution alone. In the specific framework "Variation-based Cause Effect Identification" (VCEI), the setting is bivariate causal discovery from a single observational dataset: one observes i.i.d. pairs $\mathcal{D}=\{(x_n,y_n)\}_{n=1}^N$ sampled from a joint distribution $p_{xy}$ and must decide whether $x \to y$ or $y \to x$. The framework assumes acyclicity, causal sufficiency, and existence of a causal link, and it operationalizes the Independence of Cause and Mechanism (ICM) principle by artificially constructing two settings through sample reweighting and then quantifying the stability of the fitted conditional mechanism in each direction (Salem et al., 2022).

1. Problem setting and causal premise

VCEI addresses the classic cause-effect identification problem in its most restrictive form: no interventions, no time ordering, and no conditioning on additional variables. The observational input is a single empirical joint sample, and the target is a binary orientation decision between the two possible directed relations. This makes the framework narrower than general structure learning, but it also clarifies the source of identifiability: the method does not appeal to graph sparsity, multi-node conditional independences, or multiple environments; instead, it extracts asymmetry from a single observed distribution by inducing controlled marginal variation (Salem et al., 2022).

The core causal assumption is the Independence of Cause and Mechanism principle. In the causal direction $x \to y$, the paper expresses this as

$p_{y|x} \perp p_x.$

The intended interpretation is modular: the cause distribution $p_x$ and the mechanism $p_{y|x}$ are “separate modules,” so changing the marginal over the cause should not alter the conditional mechanism that maps cause to effect. In the wrong direction, this modular independence typically fails, so perturbing the marginal tends to affect the fitted reverse-direction conditional more strongly. VCEI is built precisely to exploit this asymmetry (Salem et al., 2022).

A common misunderstanding is to treat VCEI as a method for arbitrary multivariate graph recovery. The formulation in the cited work is explicitly bivariate and assumes that one of $x\to y$ or $y\to x$ is true. This suggests a closer affinity to pairwise cause-effect identification frameworks than to unrestricted DAG discovery, even though its underlying logic belongs to a broader variation-based causal discovery tradition.

2. Artificially constructed settings and reweighted marginals

The central move in VCEI is to replace the absence of observed environments with artificially created settings. For a candidate cause variable $p_{xy}$0, the method starts from the empirical marginal

$p_{xy}$1

and constructs a weighted empirical distribution

$p_{xy}$2

with

$p_{xy}$3

Accordingly, the method does not create synthetic samples in feature space; it creates a new “setting” by reweighting the observed samples (Salem et al., 2022).

The purpose of the weighting is not merely to resample the data, but to guarantee that the marginal distribution of the candidate cause has non-negligible variations. In the causal direction, such variations are expected to have no impact on the effect generation mechanism. The identification logic is therefore comparative: if reweighting the marginal of $p_{xy}$4 leaves the fitted $p_{xy}$5 conditional relatively stable, while analogous reweighting of $p_{xy}$6 perturbs the fitted $p_{xy}$7 conditional more strongly, the direction with smaller conditional disagreement is preferred (Salem et al., 2022).

This construction is significant because it provides a practical realization of ICM from a single observational setting. A plausible implication is that VCEI can be viewed as an artificial-environment analogue of methods that exploit naturally occurring heterogeneity, except that the environments are generated by constrained reweighting rather than observed directly.

3. Kernel MMD formulation and semidefinite relaxation

To quantify how different the weighted marginal is from the original empirical marginal, the paper uses maximum mean discrepancy (MMD). For a kernel $p_{xy}$8, the squared MMD between two distributions $p_{xy}$9 and $x \to y$0 is

$x \to y$1

Its empirical biased estimate is

$x \to y$2

For the weighted marginal $x \to y$3 versus the original empirical marginal $x \to y$4, this becomes

$x \to y$5

where

$x \to y$6

This kernel formulation is one of the defining features of VCEI, because the data type is largely handled through the kernel choice (Salem et al., 2022).

The weighting vector is chosen by maximizing the discrepancy between the weighted and original marginals:

$x \to y$7

subject to

$x \to y$8

As stated in the source, this is a quadratic objective over the simplex. The objective is convex in $x \to y$9, but maximizing a convex function is not a convex optimization problem in the standard sense (Salem et al., 2022).

To obtain a tractable formulation, the paper introduces a semi-definite relaxation by defining

$y \to x$0

so that

$y \to x$1

The relaxed problem is

$y \to x$2

subject to

$y \to x$3

$y \to x$4

$y \to x$5

$y \to x$6

The objective is linear in $y \to x$7, and the constraints are convex, yielding a convex QCQP / SDP-type formulation, solvable with tools like CVXPY (Salem et al., 2022).

If the optimal $y \to x$8 is rank-one, then

$y \to x$9

and the original solution is recovered exactly. The paper notes that rank-one is not guaranteed; in that case, a heuristic estimate is

$x \to y$0

This relaxation gap is one of the method’s explicit technical limitations (Salem et al., 2022).

4. Conditional stability, disagreement scores, and decision rule

After obtaining a reweighted marginal, VCEI trains predictive models in each candidate direction. In the $x \to y$1 direction, it fits a model on the original data, $x \to y$2, and another on the reweighted joint, $x \to y$3. In the reverse direction, it analogously fits $x \to y$4 and $x \to y$5. The weights are treated as sample importance weights, so training can incorporate them directly, or via resampling (Salem et al., 2022).

The impact of the artificial variation is measured using MMD between predictions of the two fitted models on a common input distribution. The resulting scores are denoted $x \to y$6 and $x \to y$7. The decision rule is explicit: if $x \to y$8, infer $x \to y$9; otherwise infer $p_{y|x} \perp p_x.$0. The interpretation is equally explicit: smaller score = more stable conditional under reweighting = more likely causal direction (Salem et al., 2022).

The algorithm therefore proceeds symmetrically in both directions. It solves the SDR on $p_{y|x} \perp p_x.$1 to obtain $p_{y|x} \perp p_x.$2, trains conditional models in the $p_{y|x} \perp p_x.$3 direction on original and weighted data, computes $p_{y|x} \perp p_x.$4, repeats on $p_{y|x} \perp p_x.$5 to obtain $p_{y|x} \perp p_x.$6, computes $p_{y|x} \perp p_x.$7, and returns the smaller-disagreement direction. This workflow turns the abstract ICM principle into a concrete comparison of causal versus anticausal stability (Salem et al., 2022).

The paper also emphasizes a subtle methodological issue: model disagreement itself can be biased, especially for flexible learners like neural networks, since two independently trained models may disagree even on identical data. This is identified as an open issue rather than a resolved property of the framework. As a partial response, the paper suggests a trend-based alternative: instead of relying on a single score, one can study the trend of disagreement as a function of $p_{y|x} \perp p_x.$8, with the expectation that the acausal direction should show a stronger increase in disagreement as perturbations become larger (Salem et al., 2022).

5. Applicability, identifiability conditions, and limitations

A major strength of VCEI is that it is kernel-based, so the data type is largely handled through the kernel choice. The source states that this means it can, in principle, work with continuous data, discrete data, mixed data, temporal data, and embeddings for text or other structured data, as long as a suitable kernel is available. Unlike many regression-based cause-effect methods, it therefore does not require the cause and effect to live in the same simple numeric space, although the final decision rule is easiest when the spaces and kernels are comparable (Salem et al., 2022).

The identifiability profile is inherited from ICM-based methods. According to the source, VCEI works best when the causal model is asymmetric, the mechanism is not equally representable in both directions, there is sufficient nonlinearity, noise is not too dominant, and confounding is absent or limited. Conversely, several limitation cases are stated explicitly: linear Gaussian settings can make the asymmetry vanish; if the same functional forms or distributional families fit equally well in both directions, the method may not identify causality; causal sufficiency is assumed, so confounding may weaken the guarantees; and the “existence of a causal link” assumption is required, although independence tests or trend-based criteria could help relax this (Salem et al., 2022).

Practical limitations are also structural. The SDR lifts the variable to an $p_{y|x} \perp p_x.$9 matrix, so it scales poorly with sample size. To reduce cost, the authors use a subset of size $p_x$0 for the optimization. The optimization can also concentrate mass on a few samples, producing degenerate Dirac-like weights. To mitigate this, the paper suggests regularizations such as

$p_x$1

where

$p_x$2

and an additional constraint on deviation from the uniform empirical distribution,

$p_x$3

Both regularizations preserve convexity (Salem et al., 2022).

A common misconception is that variation-based methods necessarily require multiple observed environments or explicit interventions. VCEI directly contradicts that view: its defining property is single observational setting only, with variation produced by reweighting observed samples rather than by collecting heterogeneous datasets.

6. Experimental evaluation and empirical position

The evaluation reported for VCEI covers both synthetic and real data. The synthetic benchmarks include SIM, SIM-c, SIM-ln, SIM-G, and additional synthetic datasets such as AN(-s), LS(-s), and MN-U. For real-world evaluation, the paper uses the Tübingen Cause-Effect Pairs benchmark, with 108 pairs in the dataset and 103 pairs used in their experiments because they are univariate and continuous/discrete. The paper reports comparison against at least GPI, and it also references benchmark comparisons against prior methods in the literature, especially those summarized in the cause-effect benchmark paper (Salem et al., 2022).

The experimental setup is specified in detail. Data are standardized using RobustScaler. A subset of size $p_x$4 is used for the optimization. The kernel is a squared exponential kernel, with kernel lengthscale chosen by MLE with KDE and 5-fold CV. The predictive model is Exact GP. The regularization parameter is $p_x$5. All experiments run on CPU (Salem et al., 2022).

The reported conclusions are deliberately measured. VCEI is described as competitive with prior cause-effect identification methods, but the paper does not claim state-of-the-art performance on all benchmarks. Its key advantage is generality: it is less tied to data type, more robust to model class choice, and conceptually grounded in ICM via artificial variations. The paper further summarizes its main conceptual contribution as sample reweighting + kernel MMD + convex relaxation, which together make the ICM principle operational from a single observational sample (Salem et al., 2022).

These results place VCEI in a distinctive empirical position. It is not presented primarily as a benchmark-optimizing architecture; rather, it is framed as a mechanism-based identification procedure whose competitiveness supports the feasibility of the underlying causal logic.

7. Relation to heterogeneous-data and change-based causal discovery

VCEI belongs to a broader family of methods that use variation as an identifying resource, but it differs from frameworks that rely on observed heterogeneity or streams of spontaneous changes. A compact comparison is useful.

Framework	Source of variation	Scope stated in the source
VCEI	Artificially created settings by reweighting samples	Bivariate cause-effect identification from a single observational dataset
CHOD	Exogenous covariate $p_x$6 with varying direct effects $p_x$7	Heterogeneous observational data; identifiability results for several cyclic/confounded or acyclic settings
Causal Discovery from Changes	Local spontaneous mechanism changes across a transition sequence	Equivalence-class recovery from a stream of distributions

In “Causal Discovery with Heterogeneous Observational Data” (Zhou et al., 2022), the source of variation is an exogenous covariate $p_x$8 in a linear structural equation model,

$p_x$9

so causal coefficients vary with heterogeneity rather than remaining invariant. The paper studies heterogeneous observational data, allows cycles and confounders in important cases, and develops a fully Bayesian structure-learning procedure with spline parameterization, spike-and-slab priors, and MCMC. Its variation-based logic is therefore different from VCEI’s ICM-based stability criterion: CHOD does not construct artificial perturbations of a candidate cause distribution, but instead models how causal effects vary with $p_{y|x}$0 and uses those variations as causal signatures (Zhou et al., 2022).

In “Causal Discovery from Changes” (Tian et al., 2013), the variation source is a transition sequence generated by local spontaneous mechanism changes. Under modularity, locality of changes, and influentiality, the paper shows that a mechanism change at $p_{y|x}$1 may alter the marginals of descendants of $p_{y|x}$2 and cannot alter the marginals of nondescendants of $p_{y|x}$3. The resulting framework characterizes transition-pair and transition-sequence equivalence classes, constructs order graphs and marked order graphs, and can be combined with static causal discovery to recover a tighter equivalence class than is available from a single stationary distribution (Tian et al., 2013).

The conceptual connection across these frameworks is clear: all treat variation itself as evidence. The difference lies in where the variation comes from and what assumptions accompany it. VCEI generates variation by constrained reweighting within a single observational sample and asks which direction is more invariant; CHOD assumes observed or latent heterogeneity that modulates causal coefficients; Tian and Pearl assume a stream of localized mechanism changes across distributions. This suggests that “variation-based causal discovery” is best understood not as one algorithm, but as a research program in which causal asymmetry is exposed by comparing mechanisms across settings—whether those settings are artificial, heterogeneous, or dynamically changing (Salem et al., 2022).