--- title: Variation-Based Causal Discovery (VCEI) url: https://www.emergentmind.com/topics/variation-based-causal-discovery-vcei type: topic --- # Variation-Based Causal Discovery (VCEI) Variation-based causal discovery designates causal inference strategies that exploit changes, heterogeneity, or induced variation to reveal asymmetries that are not available from a single stationary distribution alone. In the specific framework "Variation-based Cause Effect Identification" (VCEI), the setting is **bivariate causal discovery from a single observational dataset**: one observes i.i.d. pairs $\mathcal{D}=\{(x_n,y_n)\}_{n=1}^N$ sampled from a joint distribution $p_{xy}$ and must decide whether $x \to y$ or $y \to x$. The framework assumes **acyclicity**, **causal sufficiency**, and **existence of a causal link**, and it operationalizes the **Independence of Cause and Mechanism (ICM)** principle by artificially constructing two settings through sample reweighting and then quantifying the stability of the fitted conditional mechanism in each direction [2211.12016]. ## 1. Problem setting and causal premise VCEI addresses the classic **cause-effect identification** problem in its most restrictive form: no interventions, no time ordering, and no conditioning on additional variables. The observational input is a single empirical joint sample, and the target is a binary orientation decision between the two possible directed relations. This makes the framework narrower than general structure learning, but it also clarifies the source of identifiability: the method does not appeal to graph sparsity, multi-node conditional independences, or multiple environments; instead, it extracts asymmetry from a single observed distribution by inducing controlled marginal variation [2211.12016]. The core causal assumption is the **Independence of Cause and Mechanism** principle. In the causal direction $x \to y$, the paper expresses this as $$ p_{y|x} \perp p_x. $$ The intended interpretation is modular: the cause distribution $p_x$ and the mechanism $p_{y|x}$ are “separate modules,” so changing the marginal over the cause should not alter the conditional mechanism that maps cause to effect. In the wrong direction, this modular independence typically fails, so perturbing the marginal tends to affect the fitted reverse-direction conditional more strongly. VCEI is built precisely to exploit this asymmetry [2211.12016]. A common misunderstanding is to treat VCEI as a method for arbitrary multivariate graph recovery. The formulation in the cited work is explicitly **bivariate** and assumes that one of $x\to y$ or $y\to x$ is true. This suggests a closer affinity to pairwise cause-effect identification frameworks than to unrestricted DAG discovery, even though its underlying logic belongs to a broader variation-based causal discovery tradition. ## 2. Artificially constructed settings and reweighted marginals The central move in VCEI is to replace the absence of observed environments with **artificially created settings**. For a candidate cause variable $x$, the method starts from the empirical marginal $$ p_{x,N}(x)=\frac{1}{N}\sum_{n=1}^N \delta_{x_n}(x), $$ and constructs a weighted empirical distribution $$ p_{x,N}^{\alpha}(x)=\sum_{n=1}^N \alpha_n \delta_{x_n}(x), $$ with $$ \alpha_n \ge 0, \qquad \sum_{n=1}^N \alpha_n=1. $$ Accordingly, the method does not create synthetic samples in feature space; it creates a new “setting” by **reweighting the observed samples** [2211.12016]. The purpose of the weighting is not merely to resample the data, but to guarantee that the marginal distribution of the candidate cause has **non-negligible variations**. In the causal direction, such variations are expected to have no impact on the effect generation mechanism. The identification logic is therefore comparative: if reweighting the marginal of $x$ leaves the fitted $x\to y$ conditional relatively stable, while analogous reweighting of $y$ perturbs the fitted $y\to x$ conditional more strongly, the direction with smaller conditional disagreement is preferred [2211.12016]. This construction is significant because it provides a practical realization of ICM from a single observational setting. A plausible implication is that VCEI can be viewed as an artificial-environment analogue of methods that exploit naturally occurring heterogeneity, except that the environments are generated by constrained reweighting rather than observed directly. ## 3. Kernel MMD formulation and semidefinite relaxation To quantify how different the weighted marginal is from the original empirical marginal, the paper uses **maximum mean discrepancy (MMD)**. For a kernel $k$, the squared MMD between two distributions $p$ and $q$ is $$ \mathrm{MMD}_k^2(p,q)=\left\|\mu_p-\mu_q\right\|_{\mathcal H}^2. $$ Its empirical biased estimate is $$ \mathrm{MMD}_k^2(p,q)\simeq \frac{1}{N^2}\sum_{i,j=1}^N k(x_i,x_j) -\frac{2}{NM}\sum_{i,j=1}^{N,M}k(x_i,y_j) +\frac{1}{M^2}\sum_{i,j=1}^M k(y_i,y_j). $$ For the weighted marginal $p_{x,N}^{\alpha}$ versus the original empirical marginal $p_{x,N}$, this becomes $$ \mathrm{MMD}_k^2(p_{x,N}^{\alpha},p_{x,N}) \simeq \alpha^\top K_{xx}\alpha -\frac{2}{N}\alpha^\top K_{xx}\mathbf{1} +\frac{1}{N^2}\mathbf{1}^\top K_{xx}\mathbf{1}, $$ where $$ K_{xx}=[k(x_i,x_j)]_{i,j=1}^N. $$ This kernel formulation is one of the defining features of VCEI, because the data type is largely handled through the kernel choice [2211.12016]. The weighting vector is chosen by maximizing the discrepancy between the weighted and original marginals: $$ \underset{\alpha}{\text{maximize} \quad \mathrm{MMD}_k^2(p_{x,N}^{\alpha},p_{x,N}) $$ subject to $$ \mathbf{1}^\top \alpha = 1, \qquad \alpha \ge 0. $$ As stated in the source, this is a **quadratic objective** over the simplex. The objective is convex in $\alpha$, but maximizing a convex function is not a convex optimization problem in the standard sense [2211.12016]. To obtain a tractable formulation, the paper introduces a **semi-definite relaxation** by defining $$ A=\alpha\alpha^\top, $$ so that $$ \alpha^\top K_{xx}\alpha = \mathrm{Tr}(AK_{xx}) = A \bullet K_{xx}. $$ The relaxed problem is $$ \underset{A}{\text{maximize} \quad A \bullet \left(K_{xx}-\frac{2}{N}K_{xx}\mathbf{1}\mathbf{1}^\top\right) +\frac{1}{N^2}\mathbf{1}^\top K_{xx}\mathbf{1} $$ subject to $$ \begin{bmatrix} A & A\mathbf{1}\\ \mathbf{1}^\top A & 1 \end{bmatrix}\succeq 0, $$ $$ A \ge 0 \quad \text{(entry-wise)}, $$ $$ \mathbf{1}^\top A \mathbf{1}=1, $$ $$ A=A^\top. $$ The objective is **linear in $A$**, and the constraints are convex, yielding a **convex QCQP / SDP-type formulation**, solvable with tools like **CVXPY** [2211.12016]. If the optimal $A^\star$ is rank-one, then $$ A^\star=\alpha^\star{\alpha^\star}^\top, $$ and the original solution is recovered exactly. The paper notes that rank-one is not guaranteed; in that case, a heuristic estimate is $$ \alpha \simeq A\mathbf{1}. $$ This relaxation gap is one of the method’s explicit technical limitations [2211.12016]. ## 4. Conditional stability, disagreement scores, and decision rule After obtaining a reweighted marginal, VCEI trains predictive models in each candidate direction. In the $x\to y$ direction, it fits a model on the original data, $\hat f_{y|x}$, and another on the reweighted joint, $\hat f_{y|x}^{\alpha}$. In the reverse direction, it analogously fits $\hat g_{x|y}$ and $\hat g_{x|y}^{\beta}$. The weights are treated as **sample importance weights**, so training can incorporate them directly, or via resampling [2211.12016]. The impact of the artificial variation is measured using **MMD between predictions of the two fitted models on a common input distribution**. The resulting scores are denoted $S_{x\to y}$ and $S_{y\to x}$. The decision rule is explicit: if $S_{x\to y}