Temporal Causal Evaluation

• Temporal Causal Evaluation is a family of frameworks that assess causal relations by capturing irreversible state changes and temporal dependencies.
• It spans diverse applications including vision-language reasoning, longitudinal treatment-effect estimation, and operator-based time-series analysis.
• The approach emphasizes separating true causal mechanisms from mere temporal order, while addressing challenges like bias and confounding.

Searching arXiv for papers and directly relevant uses of “Temporal Causal Evaluation (TCE)”. arxiv_search query="\"Temporal Causal Evaluation\" OR TimeCausality temporal causality benchmark vision LLMs" max_results=10 Temporal Causal Evaluation (TCE) is a family of evaluation frameworks concerned with causal relations that unfold over time. In recent arXiv usage, the term does not denote a single standardized protocol. Instead, it appears in several technically distinct settings: evaluation of whether vision-LLMs can infer and explain irreversible object-state changes from images, estimation and validation of causal effects of time-varying treatments under time-varying confounding, analysis of causal effects between temporal patterns in time and frequency domains, and diagnosis of multi-step causal credit assignment in formal reactive systems (Wang et al., 21 May 2025, Gupta et al., 2022, Reiter et al., 2022, Holzer et al., 31 Oct 2025). Across these settings, the common concern is not mere temporal ordering, but whether a system or method captures temporally structured cause-and-effect relations.

1. Conceptual scope and major usages

The most consistent cross-domain characterization of TCE is evaluation of causal structure in the presence of temporal dependence. In the TimeCausality benchmark, TCE focuses on whether a vision-LLM can infer and explain time-driven, irreversible changes in object states based on real-world causal knowledge, not merely detect or order events (Wang et al., 21 May 2025). In longitudinal causal inference, TCE seeks to define, identify, estimate, and validate causal effects of time-varying treatments on time-varying outcomes in the presence of time-varying confounders (Gupta et al., 2022). In time-series pattern analysis, TCE is built around causal effects between processes with respect to temporal patterns, including frequency-domain formulations and singular-value decompositions of causal operators (Reiter et al., 2022). In formal reasoning benchmarks such as TempoBench, TCE measures whether a LLM can perform multi-step causal credit assignment over time in reactive systems synthesized from temporal logic (Holzer et al., 31 Oct 2025).

Usage	What is evaluated	Representative papers
Vision-language reasoning	Irreversible temporal state change and causal explanation from images	(Wang et al., 21 May 2025)
Longitudinal causal inference	Effects of time-varying treatments under evolving confounding	(Gupta et al., 2022, Luo et al., 2022, Cheng et al., 2024)
Time-series causal operators	Interventional effects over windows, patterns, and frequencies	(Reiter et al., 2022, Kleinberg et al., 2012)
Formal reasoning benchmarks	Minimal temporally ordered causes for outputs in reactive systems	(Holzer et al., 31 Oct 2025)
Temporal graphs and predictors	Counterfactual validation, delay localization, and predictor-implied influence	(Rahman et al., 2 Feb 2026, Redden et al., 26 Mar 2026, Rahman, 26 Jun 2026)

This multiplicity of usage is itself significant. TCE is not confined to one ontology of causality: some formulations are interventional, some are counterfactual, some are design-based, and some are explicitly benchmark-oriented. A plausible implication is that the term functions as a methodological umbrella for testing whether temporal dependencies are being treated causally rather than descriptively.

2. Irreversible temporal causality in vision-LLMs

The TimeCausality benchmark operationalizes TCE for vision-LLMs by centering irreversible transformations of objects under real-world processes such as chemical spoilage, oxidation and corrosion, human aging, destructive physical change, environmental modification, and artificial processing (Wang et al., 21 May 2025). The paper defines temporal causality as the process by which a scene or object undergoes state changes over time due to underlying causal factors in the world; these changes are often one-way. This is explicitly contrasted with temporal perception benchmarks that emphasize action ordering, speed, or event sequence without requiring knowledge of the mechanisms that make some transitions irreversible.

A proposed formalization introduces an object state $S_t$, causal factors $C$, and a transition function $f$, with

$S_{t+\Delta}=f(S_t,C)$

and an irreversibility constraint for many $C$,

$P(S_t \mid S_{t+\Delta}, C)\approx 0$

for large $\Delta$ in irreversible processes. For an image pair $(I_{\text{top}}, I_{\text{bottom}})$, temporal ordering is cast as selecting the direction with higher plausibility under the causal model,

$$y^*=\arg\max_y P(S_{\text{later}} \mid S_{\text{earlier}}, C).$$

The paper marks this formalization as proposed rather than part of its original equation set.

TimeCausality contains 700 human-verified image pairs derived from COCO 2017 validation images. It uses five causal types—Physical Change (PC), Chemical Change (CC), Natural Phenomenon (NP), Environmental Modification (EM), and Artificial Processing (AP)—and constructs later states by a pipeline in which Grounded SAM detects objects, an LLM generates edit instructions, “transition reasoning,” and “inferring explanations,” GPT-4o is used as an inpainting model, and multiple annotators verify temporal order, consistency, and rationale alignment. Each sample includes an image pair, object name, type, reasoning rationales, and inferring rationales. Evaluation highlights the target object and vertically concatenates images to mitigate input-order biases.

The benchmark has three aspects. Aspect I (“Which”) tests temporal order awareness using both original and reversed image ordering. Aspect II (“Why”) asks for a free-text explanation of why the later state follows from the earlier one. Aspect III (“What”) asks what caused the transition. Aspect I reports Accuracy, ACC-R, Group Score, and

$$F1=\frac{2\cdot ACC \cdot ACC\text{-}R}{ACC+ACC\text{-}R},$$

while Aspects II and III use an LLM-as-a-judge, specifically Llama 3 8B, scoring outputs from 0 to 5 against ground-truth rationales.

The results are notable because they expose a gap that standard VQA benchmarks do not. GPT-4o attains ACC 70.86, ACC-R 64.86, Group Score 43.43, F1 67.83, Reasoning Score 2.45, and Inferring Score 2.80. Qwen2.5-VL-7B reaches ACC 41.14, ACC-R 71.14, Group Score 28.00, F1 55.13, Reasoning 2.23, and Inferring 2.29. Several models exhibit severe position bias: Llama3.2-Vision-11B has ACC 79.14 but ACC-R 1.43, and GPT-4o-mini has ACC 98.57 but ACC-R 1.43. The benchmark therefore treats ACC alone as misleading.

Model	Aspect I summary	Aspect II/III summary
GPT-4o	ACC 70.86, ACC-R 64.86, Group 43.43, F1 67.83	Reasoning 2.45, Inferring 2.80
Qwen2.5-VL-7B	ACC 41.14, ACC-R 71.14, Group 28.00, F1 55.13	Reasoning 2.23, Inferring 2.29
Llama3.2-Vision-11B	ACC 79.14, ACC-R 1.43, Group 1.43, F1 29.67	Severe position bias
GPT-4o-mini	ACC 98.57, ACC-R 1.43, Group 1.43, F1 34.56	ACC alone is misleading

The broader significance of this formulation is that TCE in multimodal systems requires more than temporal sequencing. The benchmark’s error analysis shows failures in irreversibility modeling, reliance on superficial cues, weak visual-language alignment, and sensitivity to subtle visual changes such as small mold spots or fine wrinkles. Its recommended extensions include counterfactual probes, explicit causal state models, and natural time-lapse or longitudinal imagery.

3. Longitudinal treatment-effect estimation and temporal policy evaluation

In longitudinal causal inference, TCE refers to identifying and estimating causal effects of treatments that vary over time while outcomes and confounders also evolve. DRTCI formulates the problem with histories

$C$0

potential outcomes such as $C$1, and standard identification assumptions: consistency, positivity, and sequential strong ignorability (Gupta et al., 2022). It also defines dynamic treatment regimes $C$2 and the policy value

$C$3

Temporal estimands include the step-wise average treatment effect

$C$4

and cumulative effects over horizons.

DRTCI’s central claim is architectural: the latent temporal state is disentangled into outcome-only, confounding, and treatment-only components,

$C$5

Balancing is applied only to $C$6, while treatment assignment is modeled using $C$7 and $C$8, and importance weighting is estimated from $C$9. The encoder objective is

$f$0

Empirically, on a semi-synthetic lung cancer simulator, DRTCI improves counterfactual outcome prediction relative to CRN, RMSN, and MSM, especially under high confounding and multi-step horizons. Under moderate confounding and $f$1, DRTCI reports NRMSE% values 2.01, 2.66, 3.00, 2.81, and 2.14, versus CRN’s 2.43, 2.83, 3.18, 3.51, and 3.93.

A second line of work extends TCE to dependent experiments with explicit decomposition of temporal effects. The temporal and spatio-temporal Varying Coefficient Decision Process (VCDP) model represents outcomes and state transitions with time-varying coefficients and decomposes the Average Treatment Effect into Direct Effect (DE) and Indirect Effect (IE) (Luo et al., 2022). In the temporal linear case,

$f$2

while IE accumulates carryover through state-transition matrices $f$3 and treatment effects on states $f$4. The framework emphasizes switchback designs, temporal and spatial random effects, and Wald or bootstrap inference for DE and IE.

Design-based TCE appears in single-unit time-series work as well. One framework defines context-specific causal parameters $f$5 on a single observed series, averages them over time, and estimates them using targeted maximum likelihood estimation with martingale asymptotics and double robustness (Laan et al., 2018). Another extends the potential-outcomes framework to single time-series experiments and uses Horvitz–Thompson estimators together with exact randomization-based $f$6-values for sharp nulls and conservative CLT-based inference for average effects (Bojinov et al., 2017).

When latent confounders vary over time, TDCIV introduces learned time-varying conditional instrumental variables. It uses an LSTM plus VAE to disentangle a learned instrument and conditioning set from proxies, and under the Markov property and availability of proxy variables argues that these learned representations are valid for debiasing temporal causal effect estimation (Cheng et al., 2024). In synthetic studies, TDCIV reports lower absolute error than baselines including TIFM; for example, with 3 measured covariates and $f$7, at $f$8, TDCIV obtains $f$9 versus TIFM’s $S_{t+\Delta}=f(S_t,C)$0.

4. Operator, logic, and explanation-oriented formulations

A more explicitly mathematical usage of TCE treats causal effects as operators over temporal windows or patterns. In “Causal inference for temporal patterns,” the central object is the time-windowed causal effect map

$S_{t+\Delta}=f(S_t,C)$1

with Jacobian $S_{t+\Delta}=f(S_t,C)$2 (Reiter et al., 2022). Projecting this operator onto Fourier, wavelet, or SSA bases yields causal effects for temporal patterns, including the Fourier time-windowed causal effect matrix

$S_{t+\Delta}=f(S_t,C)$3

The singular-value decomposition of $S_{t+\Delta}=f(S_t,C)$4 defines Causal Orthogonal Functions (COF), a causal analogue of mSSA modes. This framework is explicitly interventional and is contrasted with frequency-domain Granger causality: GC reflects cause-process dynamics, while FTWC quantifies effect-process responsiveness to interventions.

Another formalization uses probabilistic temporal logic. “The Temporal Logic of Causal Structures” expresses time-bounded causal relations in PCTL over discrete-time Markov chains and defines a prima facie cause through temporal priority, probability raising, and occurrence conditions (Kleinberg et al., 2012). It then measures causal significance by contextual averaging:

$S_{t+\Delta}=f(S_t,C)$5

and uses empirical Bayes local FDR to select statistically significant causes.

PSI-Miner provides a sequence-rule variant of TCE for dense-time traces. It mines temporal causal sequence explanations for a target event using decision trees, interval arithmetic, pseudo-targets, and PSI-L rules of the form

$S_{t+\Delta}=f(S_t,C)$6

(Costa et al., 2019). Its key evaluation measures are support, correlation, coverage, and Unified Error, which corrects Shannon-style impurity for overlapping temporal labels.

A related explanation-centered formulation generalizes Structural Causal Explanations to temporal settings by constructing recursive explanation trees over time-indexed variables, with retrospective and anticipative modes, context selection, and temporal sequence indicators (Rödling et al., 4 Jun 2025). The paper explicitly states that, although it does not name TCE, its temporal SCE framework can be used as a protocol for evaluating causal relations over time.

5. Diagnostic benchmarks for learned temporal systems and graphs

TempoBench defines TCE as a formally grounded diagnostic task for multi-step causal credit assignment in reactive systems synthesized from temporal logic (Holzer et al., 31 Oct 2025). Systems are represented as HOA-described finite-state controllers synthesized from LTL, and the task asks for the minimal set of temporally ordered input conditions necessary for a designated output property at a given time step. Causality is operationalized through counterfactual input variation and minimality, using the CORP tool to synthesize minimal causal automata. Evaluation uses Precision, Recall, and F1 at two granularities: F1(AP), which gives partial credit at the atomic-proposition level within each timestep, and F1(TS), which requires exact timestep-level matches. Aggregated results show strong degradation with complexity: on time-step evaluation, models score 65.6% on TCE-normal and 7.5% on TCE-hard.

Temporal link prediction has generated another TCE strand. CTIGs—causal temporal interaction graphs—are synthetic continuous-time event sequences with known ground-truth causal structure, including excitatory and inhibitory effects (Rahman et al., 2 Feb 2026). The framework proposes counterfactual validation of temporal link prediction models under controlled causal shifts and timestamp shuffling, with a causal-distance metric based on cross-model predictive error. The empirical finding is that some predictors, such as TGN, show increasing performance degradation as causal distance increases, while others, such as JODIE, remain comparatively insensitive.

Causal-INSIGHT moves from data-generating causal structure to predictor-implied temporal influence. It probes a fixed trained temporal predictor with intervention-inspired clamping at inference time, constructing an influence tensor

$S_{t+\Delta}=f(S_t,C)$7

and then selecting a sparse directed temporal graph using the Qbic criterion (Redden et al., 26 Mar 2026). The method is explicitly model-agnostic and post-hoc. It evaluates how a predictor uses temporal dependencies rather than inferring the data-generating process itself.

A further refinement appears in probabilistic temporal graph generation. “Estimation–Prediction Tradeoff in Causal Probabilistic Temporal Graphs” argues that predictive accuracy alone can conflate model error with irreducible uncertainty (Rahman, 26 Jun 2026). In its binary logistic temporal graph model, regimes that maximize Fisher information also maximize entropy, so better parameter recoverability can coincide with harder point prediction. The paper therefore advocates joint evaluation of causal parameter recovery, predictive performance, and irreducible uncertainty.

6. Limitations, controversies, and research directions

Several recurring limitations cut across the literature. In TimeCausality, coverage is intentionally restricted to five causal categories, later states are generated by inpainting with GPT-4o, and some transformations are rare or context-dependent; irreversible processes are emphasized, so reversible processes are underrepresented (Wang et al., 21 May 2025). In longitudinal representation-learning approaches such as DRTCI, the paper does not present formal theorems of unbiasedness under misspecification, positivity violations can destabilize weights, and teacher-forcing versus autoregressive rollout gaps remain a concern (Gupta et al., 2022). In temporal-pattern operators, estimation currently relies on assumptions such as stationarity, no latent confounders, and no contemporaneous coupling in the time-series graph (Reiter et al., 2022).

Benchmark-centric approaches introduce another class of limitation. TempoBench is formally verifiable but synthetic, and its evaluation currently relies on exact JSON matching rather than graded natural-language explanation (Holzer et al., 31 Oct 2025). CTIG-based validation requires access to causal parameters to compute causal distance, while constructed negatives are not guaranteed to be infeasible (Rahman et al., 2 Feb 2026). Causal-INSIGHT is explicit that its output is predictor-dependent and not necessarily identical to the data-generating graph (Redden et al., 26 Mar 2026). In probabilistic temporal graphs, the estimation–prediction tradeoff implies that low predictive accuracy can be intrinsic rather than evidentiary of poor causal learning (Rahman, 26 Jun 2026).

These differences also underlie a persistent conceptual controversy: whether TCE should evaluate causal effects in the data-generating process, causal credit assignment in a learned model, or causal plausibility of a benchmark response. The literature supports all three usages. A plausible synthesis is that robust TCE requires at least four elements: explicit temporal structure, a causal criterion stronger than sequence ordering, diagnostics that separate spurious success from genuine mechanism capture, and evaluation procedures that expose bias, inconsistency, or irreducible uncertainty. Under that synthesis, TCE is less a single benchmark than a methodological orientation toward testing temporal causality as such, rather than temporal correlation or temporal prediction alone.