Causal Process Framework Theory

• Causal Process Framework is a novel approach that models dynamic, evolving causal relationships with sparse, time-varying graphs to capture local interactions.
• It integrates neural update rules with reinforcement learning to discretely select causal connections between object and force nodes for transparent graph recovery.
• Empirical results demonstrate improved prediction accuracy and RL performance by efficiently capturing transient interactions in object-centric, multi-agent environments.

The Causal Process Framework is a novel theoretical and algorithmic approach to modeling and discovering dynamic causal relationships within systems, particularly those characterized by object-centric interactions and evolving causal dependencies over time (Orujlu et al., 18 Jul 2025). It advances traditional notions of causality by allowing for dynamically instantiated causal graphs, where connections adapt as interactions between entities emerge or dissipate. The framework is instantiated in the Causal Process Model (CPM), which reinterprets modern attention mechanisms—such as those in Transformer architectures—as a structured, reinforcement learning (RL)-based decision process aimed at inferring the latent causal structure underlying observed dynamics.

1. Dynamic Causal Process Theory

The theoretical underpinning of the framework is its departure from static, fully connected causal graphs common in classical Structural Causal Models (SCMs). Instead, it models dynamic hypotheses about causality: at each time step, only the active, local interactions are represented in the causal graph. These relationships can change over time as objects engage or disengage—such as two entities interacting only upon collision, after which their causal influence vanishes. This sparse, time-varying representation is essential for systems where causal connectivity is inherently dynamic, supporting efficient modeling and interpretable structure discovery in physical or multi-agent environments.

A key implication is that the active causal graph at time $t$, denoted $G^{(t)}$, is a sparse subgraph reflecting which entities or forces are causally “connected” at that instant. The learned structure is thus a sequence $\{G^{(t)}\}_{t=1}^T$ rather than a fixed graph $G$.

2. Model Architecture and Discrete Attention Mechanism

The Causal Process Model operationalizes the framework using two types of nodes: object nodes (encoding the state of physical entities) and force nodes (encoding interactions). Each node has a neural update function—$f_O$ for object nodes and $f_F$ for force nodes—parameterized by learned weights and sharing parameters across all nodes of the same type. The state factorization is thus:

• $\mathcal{O}^{(t)}$: set of object node states at time $t$,
• $\mathcal{F}^{(t)}$: set of force node states at time $t$.

Critically, the CPM departs from standard soft-attention mechanisms. Rather than computing full, dense attention over all node pairs with continuous weights, CPM uses policies implemented as separate RL agents to select discrete, all-or-nothing connections between nodes. For each time $t$:

• The interaction scope controller $\rho_O^{(t)}$ selects which objects participate in interactions, sampling a subset $J^{(t)}$.
• The effect attribution controller $\rho_{O\rightarrow F}^{(t)}$ selects which forces update object states, sampling $I^{(t)}$.

Mathematically, these choices are represented as categorical distributions:

$$J^{(t)} \sim \text{Categorical}(\rho_O^{(t)})\,, \quad I^{(t)} \sim \text{Categorical}(\rho_{O\rightarrow F}^{(t)}).$$

The object and force updates are then:

$$F^{(t)}_j := f_F\left(F^{(t-1)}_j, \{ O^{(t-1)}_i \mid (i,j) \in J^{(t-1)} \} \right) \ O^{(t)}_i := f_O\left(O^{(t-1)}_i, \{ F^{(t)}_j \mid (j,i) \in I^{(t-1)} \} \right)$$

This discrete edge selection provides interpretability by revealing explicit, stepwise causal connections, unlike the often opaque, softly-weighted graphs of standard attention layers.

3. RL Integration and Causal Graph Discovery

One of the most distinctive features of the framework is its integration of causal discovery into the RL paradigm. The set of possible causal graphs forms the hypothesis space, and the selection of edges (which nodes are causally connected at any time) becomes a sequential decision-making problem.

Two RL agents are employed:

• The first agent, with policy $\pi_O$, decides which objects connect to force nodes (defining the scope of interaction).
• The second agent, with policy $\pi_{O\rightarrow F}$, determines how force nodes should affect object nodes (causal attribution).

At each time step:

• The RL agents probabilistically select edges.
• The selection is trained via policy gradients, with reward functions designed to maximize downstream prediction accuracy (e.g., future state prediction, reward attainment in RL tasks).

Thus, the CPM “translates” attention into a causal graph-building process, with RL agents learning to recover interpretable and efficient causal structure directly from visual or feature-based observations.

4. Empirical Results and Comparative Performance

Empirical evaluation is conducted in synthetic physics environments that highlight dynamic object interactions (e.g., moving balls that collide). The CPM is compared against:

• A Graph Neural Network (GNN) baseline, which encodes all possible edges without discrete selection.
• A modular MLP baseline that models each object with an independent neural network.

Key findings include:

• CPM achieves higher prediction accuracy (Hits@1, Mean Reciprocal Rank) for future states, particularly over longer horizons and in environments with variable object count.
• The model generalizes robustly, outperforming baselines in environments where object properties such as mass are unobserved.
• Importantly, CPM maintains accurate and interpretable causal graphs over time, dynamically reflecting the presence or absence of interactions.

Moreover, when deployed in downstream model-based RL tasks (such as moving an object to a goal position), policies guided by CPM-generated causal graphs yield higher RL rewards compared to policies based on non-causal or static structure baselines.

5. Recovering and Interpreting Dynamic Causal Structure

A notable achievement of the framework is the explicit, time-resolved recovery of causal graphs. At each time step, the model identifies only those causal relationships that are “actualized” (such as during a collision), resulting in interpretable, sparse, and temporally-evolving causal graphs.

This is accomplished via:

• Discrete attention sampling, which inherently leads to activation/deactivation of edges as system interactions wax and wane.
• Neural update rules that are conditioned exclusively on the selected parent nodes at each step (reflecting local causality).
• By inspecting sampled graphs over time, users can directly observe when specific physical phenomena (e.g., contact forces) are instantiated or ceased in the system.

This spatiotemporal causal interpretability is unattainable with models using static or soft, continuous attention mechanisms, which tend to encode all possible dependencies indiscriminately.

6. Broader Implications and Significance

The Causal Process Framework demonstrates that embedding dynamic, discrete causal structure discovery within modern neural architectures—via RL-driven decision processes—yields practical and scientific advances:

• Interpretability: Causal graphs constructed by the model provide transparent explanations of which objects and forces are causally relevant at each step.
• Efficiency: The use of sparse, dynamically-selected graphs mitigates over-squashing and information leakage, common in dense or static approaches.
• Scalability and Generalization: The approach adapts to varying numbers and types of objects, supporting transfer to new scenarios without manual graph design.
• Integration with RL: The nested RL structure allows the system to learn causal discovery policies end-to-end, reinforcing beneficial graph-building behaviors through reward signals.

A plausible implication is that this methodology may extend to a broader range of domains requiring dynamic, interpretable causal reasoning, such as robotics, physical simulation, or multi-agent systems where causal relationships are inherently variable and local in time.

---

In summary, the Causal Process Framework provides a foundation and practical algorithms for learning, representing, and exploiting dynamic causal processes. By reframing attention as a causal graph construction task governed by RL policies, it achieves an overview of causal discovery, neural representation learning, and interpretable RL suitable for complex, object-centric environments (Orujlu et al., 18 Jul 2025).