CauSight: Visual Causal Discovery

• CauSight is a vision-language model for visual causal discovery that infers directed, predicate-labeled causal graphs over image entities rather than just detecting objects.
• It leverages the robust VCG-32K dataset—with 32K images and detailed causal annotations—to train using a pipeline that combines Tree-of-Causal-Thought synthesis with reinforcement learning.
• By integrating precise detection with high-level causal reasoning, CauSight achieves significant improvements in recall and F1 compared to baseline models.

Searching arXiv for "CauSight" to ground the article in the cited literature. CauSight is a vision-LLM for visual causal discovery, introduced together with the Visual Causal Graph dataset (VCG-32K) as a framework for inferring directed, predicate-labeled cause-and-effect relations among entities in a single image rather than merely recognizing object presence or spatial adjacency. In this formulation, an image is mapped to a causal graph over entities, and the model is trained through a pipeline that combines curated supervision, Tree-of-Causal-Thought (ToCT) trajectory synthesis, and reinforcement learning with a graph-based causal reward. The same name also appears in several later or parallel domain-specific descriptions, but the primary arXiv use of CauSight designates the visual causal discovery system introduced in late 2025 (Zhang et al., 1 Dec 2025).

1. Definition and conceptual scope

CauSight is centered on the claim that visual intelligence should not stop at identifying what is present in a scene; it should also infer what maintains the scene’s current state and what would change under intervention. The underlying task, visual causal discovery, is defined over a single image as the prediction of a directed causal graph

$G = (V, E),$

where each node is an entity with class $c$ and bounding box $b$, and each edge $(v_i, \kappa, v_j)$ denotes a predicate-labeled causal relation such that

$p(v_j \mid do(v_i = 0)) \neq p(v_j).$

Under this semantics, $v_i \to v_j$ means that intervening on $v_i$ changes the state of $v_j$ (Zhang et al., 1 Dec 2025).

This formulation differs from standard visual recognition and scene-graph parsing. Relations such as “on” or “next to” are spatial or associative; they do not specify which entity must be moved first to prevent a cascade of changes in the scene. CauSight therefore operates at what the paper calls a supersense for causality: a layer above raw detection in which the model moves from low-level identification to high-level causal dispositions. In the paper’s terminology, the system aims to answer questions such as “what will happen if I remove this?” rather than only “what is here?” (Zhang et al., 1 Dec 2025).

A plausible implication is that CauSight occupies an intermediate position between scene understanding and intervention-aware reasoning. It does not model full structural-causal dynamics over time, but it does formalize static, image-grounded interventional dependence at the entity level.

2. VCG-32K and the annotation protocol

The training and evaluation substrate for CauSight is VCG-32K, described as the first large-scale $\sim 32$K-image dataset with entity-level causal graphs. The dataset contains 32,256 total images, with images drawn from MS-COCO (11,428) and Objects365 (20,828). It includes 299,262 entities, spanning 2,287 object categories, and 185,321 directed causal edges, or approximately 5.75 per image (Zhang et al., 1 Dec 2025).

The annotation pipeline has two stages. In Stage 1, fifty annotators refine bounding boxes by correcting, deleting, or adding boxes so that each entity is in direct contact with at least one other entity. In Stage 2, annotators assign a directed predicate $\kappa$ for every ordered pair $c$0 when three conditions are satisfied: $c$1 touches $c$2 physically; the presence of $c$3 maintains the current state of $c$4; and removing $c$5 would break the state of $c$6. Ten senior reviewers continuously audit the process to keep accuracy above 95% (Zhang et al., 1 Dec 2025).

The dataset design imposes a specific notion of visual causality: causal edges are grounded in physical contact and in state maintenance under removal. This narrows the task relative to broader counterfactual reasoning, but it yields a tractable annotation protocol and a graph structure closely aligned with manipulation-relevant dependencies. The edge predicates cited in the paper include examples such as “support,” “carry_on,” and “lift,” indicating that the graph is not merely directed but also semantically labeled (Zhang et al., 1 Dec 2025).

3. Model architecture and optimization pipeline

CauSight uses Qwen2.5-VL-7B as its policy model and Qwen2.5-VL-72B as a teacher for trajectory synthesis. The backbone encodes image pixels and then autoregressively generates text tokens. The core methodological contribution is the integration of three components: training data curation from VCG-32K, ToCT for synthesizing reasoning trajectories, and reinforcement learning with a designed causal reward (Zhang et al., 1 Dec 2025).

ToCT synthesizes structured reasoning trajectories

$c$7

where each intermediate output $c$8 follows one of three actions: Region Selection $c$9, Entity Recognition $b$0, and Causality Orientation $b$1. At step $b$2, the state $b$3 records previous regions, discovered edges, and the last action, and the policy $b$4 selects actions in the order $b$5, sampling

$b$6

The search over action sequences is carried out with Monte-Carlo Tree Search, where each node stores visit counts $b$7 and state-action values $b$8, and selection follows the UCT rule

$b$9

After $(v_i, \kappa, v_j)$0 steps and up to 20 MCTS roll-outs per image, the highest-value trajectory is extracted, and only trajectories whose final recall strictly exceeds that of one-step prompting are retained (Zhang et al., 1 Dec 2025).

The filtered $(v_i, \kappa, v_j)$1 pairs are used for supervised fine-tuning with

$(v_i, \kappa, v_j)$2

Starting from this SFT policy, CauSight applies Group Relative Policy Optimization (GRPO). For a query $(v_i, \kappa, v_j)$3, with roll-outs $(v_i, \kappa, v_j)$4 and rewards $(v_i, \kappa, v_j)$5, the relative advantage is

$(v_i, \kappa, v_j)$6

and the importance ratio is

$(v_i, \kappa, v_j)$7

The objective is

$(v_i, \kappa, v_j)$8

The per-trajectory reward is

$(v_i, \kappa, v_j)$9

with

$p(v_j \mid do(v_i = 0)) \neq p(v_j).$0

and Format providing a small bonus for well-formed output (Zhang et al., 1 Dec 2025).

This training design separates three functions: ToCT constructs high-quality structured demonstrations; SFT initializes a causal reasoning policy; and GRPO directly optimizes graph-level outputs under recall- and precision-sensitive rewards. The paper’s analysis explicitly treats both ToCT and RL as critical rather than interchangeable components.

4. Benchmarking and quantitative performance

Evaluation uses graph-level Recall, Precision, and F1, with entity matching via Hungarian + GIoU $p(v_j \mid do(v_i = 0)) \neq p(v_j).$1. The in-domain benchmark consists of 350 MS-COCO images; the cross-domain benchmark consists of 350 Objects365 images; and out-of-distribution evaluation is reported on Math-V, MathVista, and BLINK. Baselines include proprietary systems—Gemini 2.5 Pro, OpenAI o3, GPT-4.1, GPT-5—and open-source systems—Qwen2.5-VL-{7B, 32B, 72B}—together with an SFT variant in which Qwen2.5-VL-7B is fine-tuned directly on formatted ground-truth labels (Zhang et al., 1 Dec 2025).

On average across the in-domain and cross-domain splits, the reported results are as follows:

Model	Recall	Precision	F1
GPT-4.1	10.2%	9.6%	9.2%
Qwen2.5-VL-72B	15.4%	23.6%	16.8%
CauSight	31.2%	45.5%	34.4%

Relative to GPT-4.1, CauSight achieves a 21 percentage-point absolute gain in Recall and a threefold improvement in F1. The abstract characterizes this as a performance gain of over threefold on visual causal discovery (Zhang et al., 1 Dec 2025).

The qualitative analysis reported in the paper contrasts three regimes. The base model produces graphs with recall approximately zero; the SFT-only variant exhibits substantial hallucination; and CauSight recovers nearly all major edges, with residual errors often attributed to bounding-box GIoU mismatches. A representative ToCT trace illustrates the recurring pattern “focus on region,” “recognize entities,” and “infer causal pairs,” which operationalizes the three-action reasoning cycle used during synthesis (Zhang et al., 1 Dec 2025).

5. Ablations, error analysis, and interpretive findings

The paper’s ablation study isolates the contribution of each training stage. Removing both ToCT and RL lowers Recall from 31.2% to 24.5%, a drop of 6.7 percentage points. Removing RL while retaining ToCT-based SFT lowers Recall to 10.1%, a drop of 21.1 percentage points. The reported interpretation is that ToCT builds a strong initial causal reasoning policy, while RL refines that policy to maximize graph-level reward (Zhang et al., 1 Dec 2025).

A second analytical axis separates reasoning quality from detection quality. The paper defines a reachable upper-bound $p(v_j \mid do(v_i = 0)) \neq p(v_j).$2: the recall obtainable if reasoning were perfect over detected nodes. It then measures the reasoning-induced loss

$p(v_j \mid do(v_i = 0)) \neq p(v_j).$3

Under this metric, GPT-4.1 has $p(v_j \mid do(v_i = 0)) \neq p(v_j).$4, Qwen2.5-VL-72B has $p(v_j \mid do(v_i = 0)) \neq p(v_j).$5, and CauSight reduces the loss to 8.0%. The paper also introduces the Recall Stability Index (RSI) to measure sensitivity of Recall to the GIoU threshold, reporting that CauSight maintains high RSI while improving reasoning performance, which is presented as evidence of a balanced integration of detection and causal understanding (Zhang et al., 1 Dec 2025).

The limitations are explicit. Extremely cluttered scenes remain challenging for both ToCT and the learned policy, including cases in which neither ToCT nor vanilla reasoning finds any edge. Bounding-box matching errors can penalize correct causal predictions when GIoU < 0.5. The method is also restricted to single-image, static causality and does not directly handle dynamic or temporal cause-effect relations (Zhang et al., 1 Dec 2025).

The future-work agenda follows directly from these limitations: extension from single images to videos for temporal causality; incorporation of explicit structural-causal models for interventions beyond object removal; more efficient synthesis of reasoning trajectories, including learned ToCT; and integration with embodied agents such as ADAM for real-world manipulation under causal constraints (Zhang et al., 1 Dec 2025).

6. Terminological range and other uses of the name

Although the principal arXiv usage of CauSight denotes the visual causal discovery model and dataset framework described above, the name also appears in several other domain-specific descriptions. In a glaucoma-detection context, a description attached to LightHCG uses “CauSight” for a pipeline combining a Convolutional VAE, HSIC-based disentanglement, a Graph Autoencoder with an acyclicity constraint, and a small classification head operating on a $p(v_j \mid do(v_i = 0)) \neq p(v_j).$6 latent subspace (Kim, 2 Dec 2025). In low-level vision, “CauSight” is used as a broader diagnostic workflow built on Causal Effect Map (CEM), extending intervention-based analysis from input patches to features, channels, conditional queries, and architecture debugging (Hu et al., 2024). In autonomous driving, the same name is used for a conceptual in-vehicle human-attention capture system derived from CUEING, emphasizing adaptive gaze-label cleansing and lightweight convolutional self-attention for on-board deployment (Liang et al., 2023).

Usage of “CauSight”	Characterization in the source	Source
Visual causal discovery	VLM for single-image causal graph prediction with VCG-32K, ToCT, and RL	(Zhang et al., 1 Dec 2025)
Glaucoma detection	LightHCG-based causal detector with CVAE, HSIC, and GAE	(Kim, 2 Dec 2025)
Low-level vision diagnostics	CEM-based causal diagnostic workflow	(Hu et al., 2024)
Driving attention capture	CUEING-derived in-vehicle human-attention system	(Liang et al., 2023)

This suggests that “CauSight” is not yet a uniquely stabilized label across preprint literature. For disambiguation, the visual causal discovery usage is identifiable by its association with VCG-32K, Tree-of-Causal-Thought, and graph-level causal prediction over image entities (Zhang et al., 1 Dec 2025).