VIGOR+: LLM–CEVAE Iterative Confounder Refinement

• The paper's main contribution is the integration of LLM-based candidate confounder generation with CEVAE validation to bridge the semantic–statistical gap in causal inference.
• It employs an iterative feedback loop where ELBO gains and alignment metrics guide the refinement of confounder proposals, ensuring measurable statistical utility.
• Empirical results indicate improved predictive accuracy and reduced bias on both synthetic and real datasets, confirming the method's practical impact.

VIGOR+ (Variational Information Gain for iterative cOnfounder Refinement) is an iterative confounder refinement framework that integrates LLM–based generation of candidate confounders and CEVAE-based statistical validation within a closed feedback loop. This methodology directly addresses the problem that in causal inference from observational data, missing or unmeasured confounders can render classical graphical modeling approaches inadequate or result in substantial bias in estimated causal effects. Earlier graph-based and proxy-based confounder adjustment methods either assume strong structural conditions, or—when incorporating domain knowledge through LLMs—do not ensure that proposed confounders have statistical utility. VIGOR+ unifies semantic confounder proposal and statistical validation into a single iterative procedure, formalizing both workflow and convergence properties, to recover confounder proxies that bridge the semantic–statistical gap (Hayete et al., 2021, Zhu et al., 22 Dec 2025).

1. Formal Problem Definition and Notation

Given $n$ i.i.d. samples, each consisting of covariates $x_i \in \mathbb{R}^d$, binary treatment $t_i \in \{0,1\}$, and outcome $y_i$, the joint data-generating process is modeled as:

$$p(x, t, y, U^*) = p(U^*)\ p(x \mid U^*)\ p(t \mid x, U^*)\ p(y \mid x, t, U^*)$$

where $U^*$ denotes the unobserved true confounder affecting both treatment and outcome. Only $(x, t, y)$ are observed. A vanilla CEVAE fits a latent variable model to infer $z \approx U^*$ in a single pass, but cannot incorporate domain-driven plausible confounders beyond what the likelihood induces.

VIGOR+ iteratively generates candidate confounders $\tilde{U}^{(k)}$ (proposed by an LLM) and evaluates their statistical contribution using CEVAE. The aim is to maximize CEVAE’s Evidence Lower Bound (ELBO) after augmenting $x$ with $x_i \in \mathbb{R}^d$0 and to enforce alignment between $x_i \in \mathbb{R}^d$1 and the learned latent $x_i \in \mathbb{R}^d$2. The optimization target is:

$x_i \in \mathbb{R}^d$3

where $x_i \in \mathbb{R}^d$4 is the ELBO gain, and $x_i \in \mathbb{R}^d$5 measures alignment (e.g., maximal Spearman correlation) (Zhu et al., 22 Dec 2025).

2. Methodology: LLM–CEVAE Iterative Feedback Loop

The core of VIGOR+ is an iterative procedure alternating between LLM-based semantic variable proposal and ELBO-driven statistical validation. The process consists of:

LLM-based Confounder Generation:

• The LLM receives the current data $x_i \in \mathbb{R}^d$6 and cumulative feedback.
• It formulates:
  • A candidate confounder name (with justification),
  • A candidate distributional form,
  • Samplewise assignments for $x_i \in \mathbb{R}^d$7, as a function of $x_i \in \mathbb{R}^d$8 and prescribed history.

CEVAE-based Validation:

• The CEVAE model is fit both on $x_i \in \mathbb{R}^d$9 (baseline) and $t_i \in \{0,1\}$0 (augmented).
• The model computes:
  • ELBO gain: $t_i \in \{0,1\}$1,
  • Alignment metrics: $t_i \in \{0,1\}$2 (maximal Spearman correlation), average mutual information $t_i \in \{0,1\}$3, and $t_i \in \{0,1\}$4.

Feedback Transformation:

• A rule-based mapping translates statistical diagnostics into natural language guidance for the next LLM round.
  • If $t_i \in \{0,1\}$5, the LLM is prompted to propose orthogonal variables.
  • If $t_i \in \{0,1\}$6, the prompt directs the LLM to focus on under-represented variable domains.
  • Previously proposed confounders are excluded from further consideration.

Iteration and Termination:

• The process continues until $t_i \in \{0,1\}$7 and $t_i \in \{0,1\}$8 exceed user-specified thresholds, a fixed number of iterations is reached, or improvements diminish below $t_i \in \{0,1\}$9 over $y_i$0 consecutive rounds.
• Pseudocode defines initialization, main loop, validation, feedback construction, and stopping criteria (Zhu et al., 22 Dec 2025).

3. Theoretical Properties and Convergence

Formal analysis establishes convergence under mild assumptions:

• Orthogonal Exploration (Lemma 1): If the LLM can generate confounders spanning orthogonal subspaces of $y_i$1, each iteration has a nonzero probability of capturing new information about $y_i$2.
• Monotonicity of ELBO (Lemma 2): Adding non-redundant confounder information to the encoder input increases the expected ELBO.
• Monotonic Improvement (Proposition): If feedback is respected, expected $y_i$3 is non-decreasing in $y_i$4.
• Convergence Theorem: Bounded ELBO and monotonic improvement guarantee the iterative sequence of confounder proposals will stabilize, producing a final $y_i$5 that delivers maximal information gain within the LLM’s semantic search space (Zhu et al., 22 Dec 2025).

A plausible implication is that the rate and quality of convergence are dictated by the diversity and expressiveness of the LLM's proposal space, and by the discriminative power of the validation metrics.

4. Relationship to Residual-based Graphical Model Approaches

Whereas VIGOR+ (Zhu et al., 22 Dec 2025) integrates semantic guidance with statistical validation, prior methods such as those in "Identification of Latent Variables From Graphical Model Residuals" (Hayete et al., 2021) extract confounder proxies strictly from residual structure of fitted graphs:

• Observed covariates $y_i$6, outcome $y_i$7, and unobserved $y_i$8.
• Residual proxies $y_i$9 are iteratively derived from graph residuals, using either PCA (linear) or autoencoder/CEVAE (nonlinear) encodings.
• The procedure alternates between structure learning (DAG estimation) and proxy inference, with formal results showing that under linearity and expansion, the principal components of the residual matrix recover $$p(x, t, y, U^*) = p(U^*)\ p(x \mid U^*)\ p(t \mid x, U^*)\ p(y \mid x, t, U^*)$$0 up to sign and scale.
• VIGOR+ (in the graphical modeling context) generalizes standard CEVAE by alternating graph and proxy updates, yielding improved identifiability under broader conditions (Hayete et al., 2021).

VIGOR+ as formulated in (Zhu et al., 22 Dec 2025) further extends this approach by leveraging LLM-driven domain knowledge, closing gaps between data-driven proxies and subject matter expertise.

5. Predictive Performance Bound and Empirical Assessment

Irrespective of confounder recovery efficacy, the improvement in predictive log-score—out-of-sample—over baseline (confounded) models is theoretically bounded. For $$p(x, t, y, U^*) = p(U^*)\ p(x \mid U^*)\ p(t \mid x, U^*)\ p(y \mid x, t, U^*)$$1 latent confounders, the maximum increase in BIC log-likelihood is $$p(x, t, y, U^*) = p(U^*)\ p(x \mid U^*)\ p(t \mid x, U^*)\ p(y \mid x, t, U^*)$$2 units; this reflects the penalty for additional covariates and encapsulates the information bottleneck induced by unobserved confounding (Hayete et al., 2021).

Experimental evaluation demonstrates:

Twins Dataset (VIGOR+ LLM–CEVAE loop, (Zhu et al., 22 Dec 2025)):

• Initial LLM confounders (“Placental Function Efficiency”) exhibit semantic plausibility but minor statistical impact ($$p(x, t, y, U^*) = p(U^*)\ p(x \mid U^*)\ p(t \mid x, U^*)\ p(y \mid x, t, U^*)$$3, $$p(x, t, y, U^*) = p(U^*)\ p(x \mid U^*)\ p(t \mid x, U^*)\ p(y \mid x, t, U^*)$$4).
• Iterative feedback elevates both: after three rounds (“Prenatal Care Quality Index”) $$p(x, t, y, U^*) = p(U^*)\ p(x \mid U^*)\ p(t \mid x, U^*)\ p(y \mid x, t, U^*)$$5, $$p(x, t, y, U^*) = p(U^*)\ p(x \mid U^*)\ p(t \mid x, U^*)\ p(y \mid x, t, U^*)$$6; ATE bias closely matches the known benchmark.

Residual-based GGM Setting (Hayete et al., 2021):

• In synthetic linear Gaussian SEMs with $$p(x, t, y, U^*) = p(U^*)\ p(x \mid U^*)\ p(t \mid x, U^*)\ p(y \mid x, t, U^*)$$7 latent variables, proxy $$p(x, t, y, U^*) = p(U^*)\ p(x \mid U^*)\ p(t \mid x, U^*)\ p(y \mid x, t, U^*)$$8 with ground-truth $$p(x, t, y, U^*) = p(U^*)\ p(x \mid U^*)\ p(t \mid x, U^*)\ p(y \mid x, t, U^*)$$9 increases from 0.49 (one iteration) to 0.51 ($U^*$05 iterations).
• RMSE for $U^*$1-parent coefficients drops substantially from 0.25 to 0.07, compared to a confounded baseline of 0.45.
• Real-data application (Huntington’s CHDI): reconstructed proxy $U^*$2 for hidden genetic marker reaches 0.47 (PCA mode) and 0.52 (autoencoder mode).

6. Implementation Details and Practical Guidance

An R package "VIGORplus" implements both the graphical model residual and autoencoder/CEVAE workflows (Hayete et al., 2021). Key features include:

• Linear-PCA Mode:
  • Structure learning via hill-climbing (bnlearn::hc), residual extraction, PCA-based proxy estimation with automatic or user-specified $U^*$3.
  • Recommended for roughly Gaussian, linear data.
• Autoencoder/CEVAE Mode:
  • Keras-based autoencoder with orthogonality/activity regularizers; dimension selection via PCA thresholding.
  • CEVAE proxy function for arbitrary nonlinear confounding; suitable for complex, high-dimensional dependencies at the cost of less stringent guarantees.
  • Batch sizes, learning rates, and patience parameters are tunable for overfitting control.
• General Practical Recommendations:
  • Monitor BIC (or ELBO) improvement; terminate when $U^*$4score $U^*$5 1.0.
  • For small $U^*$6, restrict latent/code dimension to prevent overfitting.
  • If confounding is suspected to be nonlinear or interaction-driven, prefer autoencoder/CEVAE mode and test for improvement.
• LLM–CEVAE Feedback Loop (as per (Zhu et al., 22 Dec 2025)):
  • LLM queried with feedback-augmented prompts; outputs candidate confounders in structured (e.g., JSON template) form.
  • Statistical validation returns diagnostics in numerical and natural language formats for prompt refinement.
  • Direct implementation guidance includes empirically determined convergence thresholds and rule-based feedback mapping.

7. Significance, Extensions, and Limitations

VIGOR+ operationalizes a principled bridge between knowledge-driven confounder suggestion and rigorous statistical validation. Its iterative LLM–CEVAE loop closes the gap between semantic plausibility and empirical utility. The formal convergence guarantees are predicated on the expressiveness and orthogonality diversity of the LLM proposal space and the sensitivity of ELBO-based metrics to confounder relevance.

A limitation is that, regardless of latent recovery or proxy sophistication, any gain in out-of-sample predictive power is information-theoretically capped as per the BIC bound. Furthermore, the empirical success of the approach relies on high-quality LLM feedback design and robust CEVAE optimization.

Extending beyond Gaussian and categorical models, VIGOR+ supports ordinal variables via Probability-Scale Residuals and can address monotonic or nonlinear confounding through rank transforms, kernel PCA, or deep autoencoders. The framework subsumes both purely statistical (residual-based) and semantically augmented confounder proxy pipelines, making it broadly applicable where traditional causal sufficiency assumptions are violated (Hayete et al., 2021, Zhu et al., 22 Dec 2025).