IV Co-Scientist: Multi-Agent IV Discovery

• IV Co-Scientist is a multi-agent framework for causal instrumental variable discovery that integrates large language models with domain-driven criteria.
• It decomposes the IV identification task into specialized agents (PreSelector, HumanProxy, CausalOracle, HypothesisGenerator, CriticAgents, and Grounder) to propose and validate candidate instruments.
• The system employs both statistical metrics (e.g., F-statistic, consistency metrics) and theoretical critiques to ensure instruments meet relevance, exclusion, and independence conditions.

IV Co-Scientist is a multi-agent large-language-model framework for causal instrumental variable discovery. It addresses a setting in which a treatment $T$ is endogenous with respect to an outcome $Y$, so that valid instruments must satisfy relevance, exclusion, and independence conditions that are usually justified by domain knowledge rather than identified from observational data alone. The system is introduced with a two-stage evaluation protocol: first, whether LLMs can recover canonical instruments from the literature and avoid instruments that are already regarded as invalid; second, whether a multi-agent pipeline can generate, critique, ground, and statistically contextualize candidate instruments for treatment–outcome pairs drawn from a large observational database (Sheth et al., 8 Feb 2026).

1. Causal setting and research objective

The paper formulates IV Co-Scientist in the standard causal IV setting with structural equations

$T = f(Z, U_T), \qquad Y = g(T, U_Y),$

where unobserved confounding induces endogeneity between $T$ and $Y$. A candidate instrument $Z$ must satisfy three conditions. The first is relevance,

$\mathrm{Cov}(Z,T)\neq 0.$

The second is the exclusion restriction,

$Y \perp\!\!\!\perp Z \mid T, X,$

with $X$ denoting observed covariates. The third is independence from unobserved confounding,

$Z \perp\!\!\!\perp U \mid X.$

The paper also recalls the Local Average Treatment Effect setting in its appendix, including monotonicity $Y$0 and the familiar Wald estimand

$Y$1

For estimation, the paper uses standard two-stage least squares notation,

$Y$2

$Y$3

with $Y$4 interpreted as the causal effect of $Y$5 on $Y$6 under IV validity (Sheth et al., 8 Feb 2026).

The substantive motivation is that identifying valid instruments is “interdisciplinary and context-sensitive,” often depending on institutional, economic, epidemiological, or historical reasoning. The paper therefore asks whether LLMs can help with IV discovery in three progressively harder modes: recovering well-established instruments, recognizing discredited ones, and proposing novel instruments for new treatment–outcome pairs in a real observational dataset. This framing places IV Co-Scientist squarely in the hypothesis-generation phase of causal research rather than in mechanical application of already accepted instruments.

2. Multi-agent architecture and discovery pipeline

IV Co-Scientist decomposes the task into specialized agents operating over the Gapminder database of “~500 socio-economic and health time-series variables across ~200 countries and multiple decades.” The pipeline begins with PreSelector, a purely statistical correlation filter that computes Pearson correlations between all variable pairs and retains

$Y$7

with $Y$8, while also requiring sufficient sample size. HumanProxy then acts as a semantic relevance filter, selecting pairs that are important, interpretable, and observationally studyable. CausalOracle assigns a directional hypothesis using both an LLM judgment and Granger causality; only pairs for which the LLM and Granger test agree are kept as treatment–outcome candidates $Y$9 (Sheth et al., 8 Feb 2026).

For each retained pair, HypothesisGenerator proposes $T = f(Z, U_T), \qquad Y = g(T, U_Y),$0 candidate instruments $T = f(Z, U_T), \qquad Y = g(T, U_Y),$1 and $T = f(Z, U_T), \qquad Y = g(T, U_Y),$2 possible confounders $T = f(Z, U_T), \qquad Y = g(T, U_Y),$3. The prompts position the generator explicitly “as an economist helping to identify causal relationships.” Two CriticAgents then evaluate each candidate instrument from complementary perspectives. One critic focuses on exclusion, asking whether $T = f(Z, U_T), \qquad Y = g(T, U_Y),$4 affects $T = f(Z, U_T), \qquad Y = g(T, U_Y),$5 only through $T = f(Z, U_T), \qquad Y = g(T, U_Y),$6. The other focuses on independence, asking whether $T = f(Z, U_T), \qquad Y = g(T, U_Y),$7 is independent of confounders that affect both $T = f(Z, U_T), \qquad Y = g(T, U_Y),$8 and $T = f(Z, U_T), \qquad Y = g(T, U_Y),$9. Only candidates classified as valid by both critics are retained: $T$0

Because these candidate instruments are initially conceptual rather than dataset-native, the final step is Grounder, which maps each accepted IV proposal to an actual proxy variable in the database. If no grounded proxy is found, the candidate is discarded; if no grounded IV remains for a treatment–outcome pair, generation and critique are repeated. This yields a pipeline that is not merely generative but explicitly filtered through semantic relevance, directionality, theoretical critique, and dataset grounding. The system is therefore closer to a structured research assistant than to a single-prompt causal recommender.

3. Statistical contextualization and indirect validation

The paper emphasizes that exclusion and independence are not statistically testable from observational data alone. IV Co-Scientist therefore uses statistical evidence only for contextualization, not for proof of validity. Relevance is checked numerically using the first-stage F-statistic from the regression of $T$1 on $T$2 and covariates, with the “usual rule-of-thumb” $T$3 marking a strong instrument. The more distinctive contribution is a consistency metric for settings where no ground-truth IV exists (Sheth et al., 8 Feb 2026).

Given two LLM-proposed instruments $T$4 and $T$5 for the same treatment–outcome pair, the paper defines

$T$6

It then constructs a null comparison by replacing $T$7 and $T$8 with random proxy variables $T$9 and $Y$0,

$Y$1

and normalizes the two: $Y$2 The intended interpretation is operational rather than definitive: $Y$3 indicates that LLM-proposed instruments yield more consistent causal estimates than random proxy variables. The paper motivates this with the standard IV identity

$Y$4

and the observation that valid instruments for the same causal question should converge toward the same underlying effect estimate. This does not verify exclusion or independence, but it provides a way to distinguish internally coherent instrument sets from arbitrary proxies.

A plausible implication is that IV Co-Scientist operationalizes “credibility” in two layers: a theoretical layer handled by the critics and a numerical layer handled by relevance and cross-IV consistency. The paper is explicit that the latter remains indirect evidence.

4. Recovery of canonical instruments and rejection of discredited ones

The first evaluation stage uses treatment–outcome–instrument triplets drawn from established IV literature, including military service $Y$5 earnings, education $Y$6 wages, housing $Y$7 crime, healthcare $Y$8 mortality, and migration $Y$9 wages. Recovery is scored by Exact Match (EM), based on semantic similarity using Qwen3-Embedding-0.6B, and Conceptual Match (CM), produced by a separate LLM judge scoring causal similarity on a 1–10 scale and then normalized in the results table. The strongest models—GPT-4o, o3-mini, QwQ, and Llama3.1 70B—often achieve very high CM values, with GPT-4o reporting $Z$0 for military service $Z$1 earnings and $Z$2 for education $Z$3 wages, while Llama3.1 8B is notably weaker (Sheth et al., 8 Feb 2026).

The second evaluation stage asks whether models can avoid or reject instruments already criticized in the empirical literature. Examples include rainfall for GDP $Z$4 conflict and for protests $Z$5 prices, as well as flawed Mendelian-randomization instruments for BMI $Z$6 systolic blood pressure. The paper distinguishes two failure modes. HG marks whether the proposer model suggests the flawed instrument at all. Critic marks whether the critic correctly flags it as invalid when it is injected into the candidate set. GPT-4o still proposes several flawed IVs in this benchmark, but its critic rejects most of them; weaker models sometimes avoid the flawed IVs by not generating them, yet fail more often when asked to critique them directly.

This benchmark clarifies a central point: the contribution of IV Co-Scientist is not that LLMs possess infallible IV intuition, but that proposer and critic can be decoupled. The system is therefore designed so that success does not depend on a single model never making a bad proposal; it depends on a pipeline in which criticism can veto plausible but invalid instruments.

5. Open-ended IV discovery on Gapminder

For open-ended discovery, the paper reports five treatment–outcome pairs selected by the pipeline: GDP $Z$7 Health, Income $Z$8 Emissions, Sanitation $Z$9 Mortality, Poverty $\mathrm{Cov}(Z,T)\neq 0.$0 Cholesterol, and Female literacy $\mathrm{Cov}(Z,T)\neq 0.$1 Number of kids per female. For GPT-4o, the reported first-stage F-statistics are $\mathrm{Cov}(Z,T)\neq 0.$2, $\mathrm{Cov}(Z,T)\neq 0.$3, $\mathrm{Cov}(Z,T)\neq 0.$4, $\mathrm{Cov}(Z,T)\neq 0.$5, and $\mathrm{Cov}(Z,T)\neq 0.$6, with corresponding normalized consistency values $\mathrm{Cov}(Z,T)\neq 0.$7 and $\mathrm{Cov}(Z,T)\neq 0.$8. The paper reports analogous patterns for o3-mini, QwQ, Llama3.1 70B, and Llama3.1 8B, with all reported $\mathrm{Cov}(Z,T)\neq 0.$9 values below $Y \perp\!\!\!\perp Z \mid T, X,$0 in Table 3 (Sheth et al., 8 Feb 2026).

The discovered examples are substantively interpretable. For GDP $Y \perp\!\!\!\perp Z \mid T, X,$1 Health, accepted IVs include distance to the port and global commodity prices, while colonial legal-origin dummies, fertile land, and historical settler mortality rates are rejected. For Sanitation $Y \perp\!\!\!\perp Z \mid T, X,$2 Mortality, accepted proposals include groundwater depth and sewerage investment, whereas sanitation subsidy rollout schedule, distance to health center, and terrain are rejected. For Female literacy $Y \perp\!\!\!\perp Z \mid T, X,$3 Number of kids per female, accepted proposals include number of female teachers, raised compulsory school-leaving age, and introduction year of girls-only scholarship programs, while distance to school and historical missionary girls’ schools are rejected.

The ablation study varies the number of proposed instruments $Y \perp\!\!\!\perp Z \mid T, X,$4 and confounders $Y \perp\!\!\!\perp Z \mid T, X,$5 for GDP $Y \perp\!\!\!\perp Z \mid T, X,$6 Health. With $Y \perp\!\!\!\perp Z \mid T, X,$7, the reported $Y \perp\!\!\!\perp Z \mid T, X,$8-statistic is $Y \perp\!\!\!\perp Z \mid T, X,$9 and $X$0. With the default $X$1, the values are $X$2 and $X$3. With $X$4, they become $X$5 and $X$6. The paper interprets $X$7 as a practical trade-off, while noting that more candidate instruments can improve relevance at higher compute cost.

6. Position within the AI co-scientist literature, limitations, and significance

IV Co-Scientist belongs to the emerging class of AI co-scientist systems that decompose scientific work into specialist agents for proposing, criticizing, ranking, and refining ideas. That decomposition is consistent with the broader “AI co-scientist” paradigm, where generation and critique are separated rather than collapsed into a single response (Gottweis et al., 26 Feb 2025). Its emphasis on end-to-end, realistic, multi-step scientific workflows also aligns with benchmark work arguing that scientific agents should be evaluated on open-ended tasks rather than on static factual queries alone (Panigrahi et al., 4 Jan 2026).

At the same time, IV Co-Scientist is narrower than co-scientist systems that close the loop with experiments or simulations. Systems such as MIND in materials science explicitly execute in-silico experiments and use the results to validate or revise hypotheses, whereas IV Co-Scientist remains at the level of causal hypothesis formation, theoretical critique, dataset grounding, and statistical contextualization (Ahn et al., 15 Apr 2026). This distinction matters because the system’s main output is not a validated causal claim, but a prioritized and more rigorously screened set of candidate instruments.

The paper is explicit about its limitations. It depends on large structured datasets such as Gapminder. Its quantitative criteria—first-stage relevance and $X$8—are proxy metrics rather than proofs of validity. The experiments are conducted in relatively well-understood socio-economic domains. The authors also position the system as a “co-scientist” or “thinking collaborator,” not a replacement for causal experts, because exclusion and independence still require substantive argument. A common misconception is therefore that the system automates IV identification in the strong sense. It does not. What it automates is a disciplined front end of the IV workflow: screening causal questions, proposing candidate instruments, critiquing them against common failure modes, grounding them in observed variables, and rejecting sets that behave no better than random proxies. That is a substantial shift in how LLMs can be used in causal inference, but it remains an aid to scientific judgment rather than a substitute for it (Sheth et al., 8 Feb 2026).