---
title: 'CauSight: Visual Causal Discovery'
url: https://www.emergentmind.com/topics/causight
type: topic
---

# CauSight: Visual Causal Discovery

Searching arXiv for "CauSight" to ground the article in the cited literature.
CauSight is a vision-language model for **visual causal discovery**, introduced together with the **Visual Causal Graph dataset (VCG-32K)** as a framework for inferring directed, predicate-labeled cause-and-effect relations among entities in a single image rather than merely recognizing object presence or spatial adjacency. In this formulation, an image is mapped to a causal graph over entities, and the model is trained through a pipeline that combines curated supervision, **Tree-of-Causal-Thought (ToCT)** trajectory synthesis, and reinforcement learning with a graph-based causal reward. The same name also appears in several later or parallel domain-specific descriptions, but the primary arXiv use of **CauSight** designates the visual causal discovery system introduced in late 2025 [2512.01827].

## 1. Definition and conceptual scope

CauSight is centered on the claim that visual intelligence should not stop at identifying *what is present* in a scene; it should also infer *what maintains the scene’s current state* and *what would change under intervention*. The underlying task, **visual causal discovery**, is defined over a single image as the prediction of a directed causal graph
$$
G = (V, E),
$$
where each node is an entity with class \(c\) and bounding box \(b\), and each edge \((v_i, \kappa, v_j)\) denotes a predicate-labeled causal relation such that
$$
p(v_j \mid do(v_i = 0)) \neq p(v_j).
$$
Under this semantics, \(v_i \to v_j\) means that intervening on \(v_i\) changes the state of \(v_j\) [2512.01827].

This formulation differs from standard visual recognition and scene-graph parsing. Relations such as “on” or “next to” are spatial or associative; they do not specify which entity must be moved first to prevent a cascade of changes in the scene. CauSight therefore operates at what the paper calls a **supersense for causality**: a layer above raw detection in which the model moves from low-level identification to high-level causal dispositions. In the paper’s terminology, the system aims to answer questions such as “what will happen if I remove this?” rather than only “what is here?” [2512.01827].

A plausible implication is that CauSight occupies an intermediate position between scene understanding and intervention-aware reasoning. It does not model full structural-causal dynamics over time, but it does formalize static, image-grounded interventional dependence at the entity level.

## 2. VCG-32K and the annotation protocol

The training and evaluation substrate for CauSight is **VCG-32K**, described as the first large-scale \(\sim 32\)K-image dataset with entity-level causal graphs. The dataset contains **32,256 total images**, with images drawn from **MS-COCO (11,428)** and **Objects365 (20,828)**. It includes **299,262 entities**, spanning **2,287 object categories**, and **185,321 directed causal edges**, or approximately **5.75 per image** [2512.01827].

The annotation pipeline has two stages. In **Stage 1**, fifty annotators refine bounding boxes by correcting, deleting, or adding boxes so that each entity is in direct contact with at least one other entity. In **Stage 2**, annotators assign a directed predicate \(\kappa\) for every ordered pair \((v_i, v_j)\) when three conditions are satisfied: \(v_i\) touches \(v_j\) physically; the presence of \(v_i\) maintains the current state of \(v_j\); and removing \(v_i\) would break the state of \(v_j\). Ten senior reviewers continuously audit the process to keep accuracy above **95%** [2512.01827].

The dataset design imposes a specific notion of visual causality: causal edges are grounded in **physical contact** and in **state maintenance under removal**. This narrows the task relative to broader counterfactual reasoning, but it yields a tractable annotation protocol and a graph structure closely aligned with manipulation-relevant dependencies. The edge predicates cited in the paper include examples such as “support,” “carry_on,” and “lift,” indicating that the graph is not merely directed but also semantically labeled [2512.01827].

## 3. Model architecture and optimization pipeline

CauSight uses **Qwen2.5-VL-7B** as its policy model and **Qwen2.5-VL-72B** as a teacher for trajectory synthesis. The backbone encodes image pixels and then autoregressively generates text tokens. The core methodological contribution is the integration of three components: training data curation from VCG-32K, ToCT for synthesizing reasoning trajectories, and reinforcement learning with a designed causal reward [2512.01827].

ToCT synthesizes structured reasoning trajectories
$$
\tau = r_1 \oplus r_2 \oplus \dots \oplus r_T,
$$
where each intermediate output \(r_t\) follows one of three actions: **Region Selection** \((A_r)\), **Entity Recognition** \((A_e)\), and **Causality Orientation** \((A_c)\). At step \(t\), the state \(s_t\) records previous regions, discovered edges, and the last action, and the policy \(\pi_\theta\) selects actions in the order \(A_r \to A_e \to A_c \to \dots\), sampling
$$
r_t \sim \pi_\theta(\cdot \mid s_t, a_t).
$$
The search over action sequences is carried out with Monte-Carlo Tree Search, where each node stores visit counts \(N(s,a)\) and state-action values \(Q(s,a)\), and selection follows the UCT rule
$$
UCT(s,a) = Q(s,a) + w \sqrt{\frac{\ln N(parent(s))}{N(s,a)}}.
$$
After **\(T=12\) steps** and up to **20 MCTS roll-outs per image**, the highest-value trajectory is extracted, and only trajectories whose final recall strictly exceeds that of one-step prompting are retained [2512.01827].

The filtered \((q,\tau)\) pairs are used for **supervised fine-tuning** with
$$
L_{SFT}(\theta) = - \mathbb{E}_{(q,\tau)}[\log \pi_\theta(\tau \mid q)].
$$
Starting from this SFT policy, CauSight applies **Group Relative Policy Optimization (GRPO)**. For a query \(q\), with roll-outs \(\{o_i\}\) and rewards \(r_i\), the relative advantage is
$$
A_i = \frac{r_i - \mathrm{mean}_j\, r_j}{\mathrm{std}_j(r_j)},
$$
and the importance ratio is
$$
R_i = \frac{\pi_\theta(o_i \mid q)}{\pi_{\theta_{\mathrm{old}}}(o_i \mid q)}.
$$
The objective is
$$
J_{GRPO}(\theta) =
\mathbb{E}_{q,\{o_i\}}
\left[
\frac{1}{G}\sum_i
\min\left(R_i A_i,\; \mathrm{clip}(R_i,1-\epsilon,1+\epsilon)A_i\right)
\right].
$$
The per-trajectory reward is
$$
R(\tau)=\lambda_r\,Recall(\tau)+\lambda_p\,Precision(\tau)+\lambda_f\,Format(\tau),
$$
with
$$
Recall = \frac{|E_{pred}\cap E_{gt}|}{|E_{gt}|}, \qquad
Precision = \frac{|E_{pred}\cap E_{gt}|}{|E_{pred}|},
$$
and **Format** providing a small bonus for well-formed output [2512.01827].

This training design separates three functions: ToCT constructs high-quality structured demonstrations; SFT initializes a causal reasoning policy; and GRPO directly optimizes graph-level outputs under recall- and precision-sensitive rewards. The paper’s analysis explicitly treats both ToCT and RL as critical rather than interchangeable components.

## 4. Benchmarking and quantitative performance

Evaluation uses **graph-level Recall, Precision, and F1**, with entity matching via **Hungarian + GIoU \(\ge 0.5\)**. The in-domain benchmark consists of **350 MS-COCO images**; the cross-domain benchmark consists of **350 Objects365 images**; and out-of-distribution evaluation is reported on **Math-V, MathVista, and BLINK**. Baselines include proprietary systems—**Gemini 2.5 Pro, OpenAI o3, GPT-4.1, GPT-5**—and open-source systems—**Qwen2.5-VL-\{7B, 32B, 72B\}**—together with an SFT variant in which Qwen2.5-VL-7B is fine-tuned directly on formatted ground-truth labels [2512.01827].

On average across the in-domain and cross-domain splits, the reported results are as follows:

| Model | Recall | Precision | F1 |
|---|---:|---:|---:|
| GPT-4.1 | 10.2% | 9.6% | 9.2% |
| Qwen2.5-VL-72B | 15.4% | 23.6% | 16.8% |
| CauSight | 31.2% | 45.5% | 34.4% |

Relative to GPT-4.1, CauSight achieves a **21 percentage-point absolute gain in Recall** and a **threefold improvement in F1**. The abstract characterizes this as a performance gain of over threefold on visual causal discovery [2512.01827].

The qualitative analysis reported in the paper contrasts three regimes. The base model produces graphs with recall approximately zero; the SFT-only variant exhibits substantial hallucination; and CauSight recovers nearly all major edges, with residual errors often attributed to bounding-box GIoU mismatches. A representative ToCT trace illustrates the recurring pattern “focus on region,” “recognize entities,” and “infer causal pairs,” which operationalizes the three-action reasoning cycle used during synthesis [2512.01827].

## 5. Ablations, error analysis, and interpretive findings

The paper’s ablation study isolates the contribution of each training stage. Removing both ToCT and RL lowers Recall from **31.2%** to **24.5%**, a drop of **6.7 percentage points**. Removing RL while retaining ToCT-based SFT lowers Recall to **10.1%**, a drop of **21.1 percentage points**. The reported interpretation is that ToCT builds a strong initial causal reasoning policy, while RL refines that policy to maximize graph-level reward [2512.01827].

A second analytical axis separates **reasoning quality** from **detection quality**. The paper defines a reachable upper-bound \(Recall_R\): the recall obtainable if reasoning were perfect over detected nodes. It then measures the reasoning-induced loss
$$
L = \frac{Recall_R - Recall}{Recall_R}.
$$
Under this metric, **GPT-4.1** has \(L = 27.9\%\), **Qwen2.5-VL-72B** has \(L = 12.9\%\), and **CauSight** reduces the loss to **8.0%**. The paper also introduces the **Recall Stability Index (RSI)** to measure sensitivity of Recall to the GIoU threshold, reporting that CauSight maintains high RSI while improving reasoning performance, which is presented as evidence of a balanced integration of detection and causal understanding [2512.01827].

The limitations are explicit. **Extremely cluttered scenes** remain challenging for both ToCT and the learned policy, including cases in which neither ToCT nor vanilla reasoning finds any edge. **Bounding-box matching errors** can penalize correct causal predictions when **GIoU < 0.5**. The method is also restricted to **single-image, static causality** and does not directly handle dynamic or temporal cause-effect relations [2512.01827].

The future-work agenda follows directly from these limitations: extension from single images to **videos for temporal causality**; incorporation of **explicit structural-causal models** for interventions beyond object removal; more efficient synthesis of reasoning trajectories, including **learned ToCT**; and integration with **embodied agents such as ADAM** for real-world manipulation under causal constraints [2512.01827].

## 6. Terminological range and other uses of the name

Although the principal arXiv usage of **CauSight** denotes the visual causal discovery model and dataset framework described above, the name also appears in several other domain-specific descriptions. In a glaucoma-detection context, a description attached to **LightHCG** uses “CauSight” for a pipeline combining a **Convolutional VAE**, **HSIC-based disentanglement**, a **Graph Autoencoder** with an acyclicity constraint, and a small classification head operating on a \(Z_2\) latent subspace [2512.02437]. In low-level vision, “CauSight” is used as a broader diagnostic workflow built on **Causal Effect Map (CEM)**, extending intervention-based analysis from input patches to features, channels, conditional queries, and architecture debugging [2407.19789]. In autonomous driving, the same name is used for a conceptual in-vehicle human-attention capture system derived from **CUEING**, emphasizing adaptive gaze-label cleansing and lightweight convolutional self-attention for on-board deployment [2305.15710].

| Usage of “CauSight” | Characterization in the source | Source |
|---|---|---|
| Visual causal discovery | VLM for single-image causal graph prediction with VCG-32K, ToCT, and RL | [2512.01827] |
| Glaucoma detection | LightHCG-based causal detector with CVAE, HSIC, and GAE | [2512.02437] |
| Low-level vision diagnostics | CEM-based causal diagnostic workflow | [2407.19789] |
| Driving attention capture | CUEING-derived in-vehicle human-attention system | [2305.15710] |

This suggests that **“CauSight” is not yet a uniquely stabilized label across preprint literature**. For disambiguation, the visual causal discovery usage is identifiable by its association with **VCG-32K**, **Tree-of-Causal-Thought**, and graph-level causal prediction over image entities [2512.01827].

Source: https://www.emergentmind.com/topics/causight