Papers
Topics
Authors
Recent
Search
2000 character limit reached

A Chosen Future Can Still Be Rewritten: Causal Writability in Video Models

Published 14 Sep 2026 in cs.LG and cs.CV | (2609.15980v1)

Abstract: When a video model generates physically incorrect motion, did it fail to learn the correct motion, or did it learn it but fail to use it? We show the latter: the correct motion remains available inside the model and can still be made to control the generated video. We train on videos where red masses oscillate slowly and blue masses oscillate quickly, then test a red mass with fast observed motion. Even when the model generates slow motion in this conflicting case, a low-dimensional edit predicted from simple physical variables restores the correct fast motion. We call this ability causal writability. At fixed strength, we find a sharp depth boundary: the same edit changes the video before the boundary but not after it. This closure marks commitment for that write. The motion signal nevertheless remains, and a stronger downstream write can restore physical motion, while excessive gain overshoots. Early causal writability predicts which errors training later corrects: those errors are writable at more network depths than errors that persist. We reproduce both causal writability and its sharp closure in a pretrained 1.3B video model, supporting generality across model scale and training regime.

Summary

  • The paper evaluates the internal representation and intervention for controllable video motion.
  • The researchers found that a compact physical edit could restore the alternative
  • Training history and observed motion affect commitment boundaries and writability

Problem formulation and central claim

The paper addresses a distinction that ordinary behavioral evaluation cannot resolve: when a video model generates physically incorrect motion, is the correct dynamics absent from its internal representation, or is it represented but causally unable to control the output? “A Chosen Future Can Still Be Rewritten: Causal Writability in Video Models” (2609.15980) isolates this distinction through controlled counterfactual video generation.

The authors construct a deliberately underspecified learning problem. During training, red spring–mass systems always oscillate slowly, while blue systems always oscillate quickly. Consequently, both observed motion and appearance support accurate prediction on the training distribution. At test time, the authors introduce unseen color–motion combinations, such as a red mass exhibiting fast motion. These conflict cases distinguish a physically grounded rule from an appearance-based shortcut. The model frequently selects the appearance-consistent continuation—for example, slow motion for a red mass—even though the observed trajectory supports fast motion.

The paper’s principal claim is that the physically correct continuation often remains internally available after the model has selected the shortcut future. A suitably structured activation intervention can redirect the decoded video toward the motion-consistent future. The authors call this property causal writability: the ability of an internal edit to change a specified aspect of the decoded output. Causal writability is not equivalent to representation or decodability. A physical variable may remain measurable in the network while losing the ability to causally affect the generated continuation.

The paper further argues that writability is depth-dependent. At a fixed intervention strength, an edit can change the decoded future at early network sites but becomes ineffective after a relatively sharp boundary. This boundary is interpreted as commitment for the particular write being tested. The physical signal need not disappear at commitment; rather, the downstream computation may fail to transmit it strongly enough into future-token states.

[Figure 1]

Figure 1: Natural generation can select one future while a causally usable physical alternative remains writable.

Experimental design

The primary experiments use a 488M-parameter latent flow-matching DiT with 30 bidirectional Transformer blocks and a frozen Wan2.1 VAE. Videos have resolution 128×128128 \times 128 and are sampled at 20 fps. The first 65 frames condition generation, and the following 64 frames constitute the predicted future. The training set contains 2,048 examples: 1,024 red–slow trajectories with angular frequency sampled from [2.2,3.0][2.2,3.0] rad/s and 1,024 blue–fast trajectories with frequency sampled from [5.2,6.4][5.2,6.4] rad/s. No conflicting color–frequency combinations occur during training.

Two history regimes test the role of evidence. Long exposes the complete observed trajectory, whereas Short replaces the first 57 frames with a fixed background and leaves only the final eight frames containing real motion. The models therefore have identical architecture, tokenization, optimization, and prediction horizon but receive different quantities of visible motion.

Evaluation is performed on fully decoded videos rather than latent activations. A deterministic detector tracks the generated mass, fits its trajectory to a sinusoidal model, and classifies the future as physics-following, shortcut-following, compromise, or off-family. This is important methodologically: the central claims concern causal effects on rendered motion, not merely linear probe accuracy or changes in latent statistics.

For mechanistic analysis, the authors construct strict aligned–conflict pairs. Each pair has identical physical state, boundary position and velocity, renderer, generation seed, and observed motion. Only the mass color differs. The aligned input uses the training-associated color and produces the motion-consistent future; the conflict input uses the opposite color and produces the shortcut continuation. Interventions modify only observed-frame condition tokens at a selected DiT site. The future-token suffix is unchanged at injection and evolves normally thereafter.

Appearance and observed motion compete during natural generation

Hue sweeps show that the model does not implement a single fixed causal rule. With observed motion held constant, changing the color cue shifts the generated future between the two training-supported motion bands. Strong endpoint colors tend to select the associated frequency, whereas ambiguous colors allow observed motion to exert greater influence.

The result is not merely a correlation between generated hue and frequency. The model also uses motion to reconstruct the appearance associated with that motion. Under ambiguous cues, fast observed motion tends to produce a blue future and slow observed motion tends to produce a red future. Thus the learned representation supports coupled appearance–dynamics modes, and either cue can partially complete the other. This is a stronger claim than simple shortcut reliance: the model has learned a joint solution space in which both motion and appearance participate in selecting the continuation.

Visible motion materially changes the competition. Long-history models remain motion-consistent farther toward the conflicting color endpoint than Short-history models. In the paper’s interpretation, additional motion evidence does not merely improve an external classifier; it delays the point at which the model commits to an appearance-conditioned future.

The same pattern recurs across 15 independently trained models, although the transition widths and the proportions of compromise outcomes vary across seeds. This seed dependence is relevant because it shows that identical training support does not determine a unique internal or behavioral solution. Nevertheless, the evidence-dependent competition between appearance and observed motion is recurrent.

[Figure 2]

Figure 2: Appearance and observed motion compete to determine the generated future.

A compact physical edit restores the alternative future

The causal test begins with a full activation difference between an aligned run and its color-conflicting counterpart. Adding the aligned-minus-conflict condition-state difference to the conflict run redirects its decoded future toward the aligned frequency. Because the two runs differ only in color, and because the intervention is evaluated on decoded videos, this establishes that a physical alternative remains causally usable inside the model.

The authors then examine whether the intervention requires a high-dimensional, trajectory-specific activation pattern. An uncentered PCA of paired differences reveals a compact structure. Retaining the first four coordinates preserves most of the effect: across three independent Short models, 87.8% of held-out Top-4 interventions fall within the prescribed near-full-recovery interval, compared with 92.4% for the full paired-difference intervention.

The coordinates are organized by the physical boundary state. For harmonic motion, position and velocity can be represented by a phase variable, and the Top-4 activation coordinates vary smoothly with this phase. A target-specific first-harmonic controller using boundary phase and requested rewrite direction predicts the edit without access to the held-out aligned activation. Its held-out coordinate R2R^2 reaches approximately $0.83$–$0.88$, compared with $0.51$–$0.60$ for a direction-only baseline. The synthesized controller achieves 85.9% near-full recovery across the six run–direction groups.

This result substantially strengthens the causal interpretation. The intervention is not simply a donor-specific patch copied from a favorable rollout. It can be predicted from a compact physical description of the input state and a target motion family. However, the demonstrated decoded evaluations are restricted primarily to the slow and fast target families. The paper does not establish reliable continuous control over unsupported intermediate frequencies.

The edit also changes future appearance in the expected direction: slow-to-fast interventions move the generated color toward blue, and fast-to-slow interventions move it toward red. This coupling indicates that the intervention accesses a joint appearance–dynamics mode rather than an isolated frequency coordinate.

The same structure transfers across independently trained models. Despite approximately 22 percentage points of variation in natural physics-following rates, a four-dimensional aligned coordinate system can be mapped across runs, and all six directed transfers achieve held-out normalized frequency recovery between $0.94$ and $0.99$. Within-direction matched coordinate correlations range from [2.2,3.0][2.2,3.0]0 to [2.2,3.0][2.2,3.0]1, while trajectory-shuffled controls yield only [2.2,3.0][2.2,3.0]2–[2.2,3.0][2.2,3.0]3. The implication is not that the models are identical, but that they can implement different downstream uses of a related physical correction space.

[Figure 3]

Figure 3: A compact, physically structured activation edit restores the motion-consistent future.

Writability closes with depth

The paper defines layerwise writability as the fraction of strict shortcut failures for which an intervention produces valid, near-aligned decoded motion at a given network site. Integrated writability is the number of writable sites across a 31-site scan, including the pre-block residual and post-block states.

At early sites, the matched edit redirects the shortcut continuation. After a narrow depth range, the same edit ceases to affect decoded motion. This sharp transition contrasts with a gradual attenuation of intervention efficacy across all layers. The authors interpret the transition as a localized commitment boundary for condition-side writing.

Training changes both behavior and writability. Across 15 Short runs, physics-following behavior increases from [2.2,3.0][2.2,3.0]4 at 5K steps to [2.2,3.0][2.2,3.0]5 at 100K. At the same time, mean writability among remaining shortcut errors decreases from 10.88 to 9.55 writable sites. Thus training can improve aggregate accuracy while making some unresolved failures more causally rigid. This is a notable and potentially contradictory result: a model may become more accurate overall while the errors it retains become less amenable to the tested physical correction.

Early writability predicts later error fate. Among trajectories that follow the shortcut at 5K, those later rescued by training are writable at 3.79 more sites than trajectories that remain failures, with a 95% confidence interval of [2.2,3.0][2.2,3.0]6. The effect survives alternative writability definitions and controls for natural error severity and boundary state. The result is predictive rather than merely descriptive: before later behavioral divergence, the depth at which a correction remains effective distinguishes errors likely to be corrected from persistent failures.

The authors control for changing error-bank composition by evaluating the same persistent-error identities at 5K and 100K. Thirteen of 15 seed cohorts show declining writability, with a mean paired change of [2.2,3.0][2.2,3.0]7 sites and a seed-clustered 95% confidence interval of [2.2,3.0][2.2,3.0]8. This makes it unlikely that the training-time contraction is solely an artifact of evaluating different examples at different checkpoints.

More observed history delays closure. In the most explicit fast-target comparison, the Long operational boundary occurs at B16, B8, and B10 for the three runs, compared with B3, B6, and B6 for Short. In Pendulum, the integrated writability increase from Long relative to Short is 10.53 sites for one target direction and 8.05 sites for the other, while the midpoint closure location shifts by 9.46 and 7.95 sites. These effects support the view that evidence strength affects how long an alternative future remains causally writable.

[Figure 4]

Figure 4: Causal writability closes with depth and predicts which early failures training later corrects.

Training order shifts commitment

The authors test whether learning cue-independent dynamics before exposing the model to appearance–dynamics correlations changes the commitment boundary. They adapt a pretrained 1.3B Wan video DiT using two paths. Direct adaptation immediately trains on the red–slow/blue–fast biased data. Neutral-first adaptation first trains on achromatic videos containing the same dynamics, then receives the biased data.

At 10K biased-stage updates, conflict physics-following is only 9.4% for Direct adaptation but 80.5% for Neutral-first adaptation on the same 128 held-out histories. Aligned performance is comparable. The result indicates that training order changes which solution controls the output, even when the subsequent biased data and adaptation procedure are matched.

Both paths exhibit sharp closure, but Neutral-first adaptation closes later: Direct loses effective writability between blocks B12 and B13, whereas Neutral-first retains it until the B14–B15 transition. The pretrained replication therefore supports two claims simultaneously: causal writability is not confined to a model trained from scratch, and the commitment boundary depends on the acquisition history of the representation.

The effect is not permanent across optimization. On fixed identities, access can close and later reopen across nearby checkpoints. Consequently, commitment is a forward-pass-local property of a particular model state and intervention, not an absorbing state of the training trajectory.

[Figure 5]

Figure 5: Learning cue-independent dynamics first improves physical behavior and delays commitment in a pretrained video DiT.

Downstream attention mediates the final write

A central mechanistic question is whether the alternative physical signal disappears after observed-frame edits lose behavioral efficacy. The authors answer negatively. When an earlier writable edit is introduced, its alternative-motion projection remains measurable in later activations through the final block, even after direct condition-side interventions no longer change decoded frequency.

Component restoration experiments localize the effective route to self-attention. Restoring natural-conflict future-token states removes frequency recovery, whereas restoring the subsequent MLP output preserves it. Within attention, restoring queries has little effect, while restoring observed-frame keys and values removes recovery. Across six run–direction groups, the difference in normalized recovery for restoring conflict K+V relative to the intact edit is approximately [2.2,3.0][2.2,3.0]9; restoring Q changes recovery by only about [5.2,6.4][5.2,6.4]0.

The interpretation is that future-frame queries continue to retrieve motion-controlling information from observed-frame K/V representations. The physical alternative survives closure, but its attention-mediated write into future-token states is insufficiently strong to control the decoded continuation.

Amplifying this downstream route can restore behavioral control. In one run, increasing the selected observed-frame V-head direction by a gain of 8 raises the physics-following rate to 56.3% across 48 held-out strict failures. A gain of 16 overshoots: outputs remain valid but mostly leave the supported motion families. This nonmonotonic response shows that causal access is not equivalent to arbitrary amplification; the intervention must remain within a locally supported regime.

The timing of the intervention also matters. Early-only flow-matching writes produce an internal response without decoded recovery, whereas late or all-call writes restore motion. Thus the relevant control problem has two axes: network depth and denoising-time allocation. A representation can be present and causally reachable in activation space yet fail to alter the final video if injected at an ineffective stage of the iterative flow-matching trajectory.

[Figure 6]

Figure 6: The alternative-motion signal survives closure, and self-attention determines whether it changes the video.

Replication across dynamical systems

The paper tests whether its conclusions depend on the sinusoidal spring task. Pendulum reproduces the competition between appearance and observed motion, the compact state-organized edit geometry, donor-free decoded recovery, and depth-localized closure. Boundary angle and angular velocity organize the Top-4 coordinates with held-out phase-plane [5.2,6.4][5.2,6.4]1 values of [5.2,6.4][5.2,6.4]2 and [5.2,6.4][5.2,6.4]3 for the two target directions. Across 64 held-out receivers, the predicted controller attains median normalized recovery of [5.2,6.4][5.2,6.4]4, close to [5.2,6.4][5.2,6.4]5 for the Top-4 oracle projection.

Free Fall extends the analysis to non-oscillatory dynamics. In this setting, a compact controller parameterized by target gravity achieves median held-out normalized recovery of [5.2,6.4][5.2,6.4]6, with gravity-fit success on 61 of 64 held-out pairs and mean absolute gravity error of [5.2,6.4][5.2,6.4]7. A rank-2 PCA representation retains 99.545% of fit-set residual energy, and direction-specific target-gravity laws achieve held-out coordinate [5.2,6.4][5.2,6.4]8 values of [5.2,6.4][5.2,6.4]9 and R2R^20.

Free Fall also displays an especially early closure boundary. Full-edit gravity success is 100% through the first two intervention sites and falls below the 75% criterion after block 1. This supports the broader claim that closure is not specific to oscillatory phase representations, although its location and mechanism are task-dependent.

[Figure 7]

Figure 7: Decoded Free Fall gravity follows the observed-motion family.

[Figure 8]

Figure 8: Free Fall reproduces localized causal writability, with direct control concentrated in early blocks.

Limitations and open questions

The strongest evidence comes from synthetic environments with exact physical variables, matched counterfactual pairs, deterministic rendering, and specialized decoded evaluators. These controls enable unusually precise causal tests but also limit external validity. The paper does not establish that naturally occurring shortcuts in broadly pretrained world models exhibit the same closure structure.

Causal writability is operationally defined relative to a particular edit, intervention site, gain, flow-matching schedule, and decoded metric. Closure therefore means that the tested write no longer changes the tested output under the tested protocol. It does not prove that no alternative intervention, parameterization, attention pathway, or downstream control mechanism remains effective.

The cross-model transfer results indicate shared edit geometry, but the implementation is not uniform. Different runs rely on different K/V heads or combinations of components. Consequently, the common structure is best interpreted as shared route semantics rather than a uniquely localized circuit.

The donor-free controllers also have restricted contracts. The Spring controller uses boundary state and target direction, while the Free Fall controller specifies target gravity. The reported decoded successes concern matched target families; they do not demonstrate robust control over arbitrary unsupported physical parameters. In addition, one auxiliary parameterization estimates the naturally generated conflict state from the first generated future, making it retrospective rather than purely anticipatory.

Finally, early writability predicts near-term training outcomes, but its predictive power attenuates over longer horizons. The paper leaves open whether a more general writability measure can forecast optimization outcomes across substantially longer training intervals, and whether the same distinction between retained information and lost causal access can be measured in natural video distributions without explicit physical state labels.

Conclusion

The paper establishes a distinction between learning a physical solution and retaining the ability to use that solution during generation. In controlled video models, appearance-based shortcuts can determine the natural future even while motion-consistent dynamics remain encoded in a compact, physically structured activation subspace. These alternatives are causally writable at early network depths, but a fixed intervention can lose efficacy at a sharp commitment boundary.

The evidence connects this boundary to the downstream attention write from observed-frame K/V states into future-frame tokens. More visible motion and cue-independent pretraining delay closure; early writability predicts which shortcut failures training later corrects. Replication in Pendulum, Free Fall, and a pretrained 1.3B video DiT indicates that the acquisition–control distinction is not restricted to a single architecture or oscillator. The specific open question is whether naturally occurring failures in large-scale video models admit equally precise measurements of causal writability, closure, and downstream recovery.

Paper to Video (Beta)

No one has generated a video about this paper yet.

Sign Up to Generate All Videos Subscribe on YouTube

Whiteboard

No one has generated a whiteboard explanation for this paper yet.

Sign Up to Generate

Explain it Like I'm 14

1. What is this paper about?

This paper studies how video-generation AI models decide what will happen next in a video.

The researchers ask an important question:

When an AI model creates physically wrong motion, does it not understand the correct motion, or does it understand it but choose not to use it?

For example, imagine an AI sees a red object moving quickly. During training, it only saw:

  • red objects moving slowly, and
  • blue objects moving quickly.

When it sees a red object moving quickly, the model must choose between two clues:

  • the color suggests slow motion;
  • the observed movement suggests fast motion.

The researchers found that the model may choose the color-based answer and generate slow motion. However, the correct fast-motion answer is still stored inside the model and can be brought back using a carefully designed internal adjustment.

They call this ability causal writability.

2. What questions did the researchers investigate?

The paper focuses on several main questions:

  1. Does the model learn the correct physical motion? Or does it simply copy an easier clue, such as an object’s color?
  2. Can the correct motion be brought back after the model chooses the wrong answer?
  3. At what point in the model’s processing does the future become difficult to change?
  4. Can researchers predict which mistakes will later be fixed through training?
  5. Does the same idea work in larger, already-trained video models?

3. How did the researchers study this?

The spring-and-mass videos

The researchers created simple videos showing a colored mass attached to a spring. The mass moved back and forth, like a toy hanging from a spring.

During training, the model saw only two combinations:

Color Movement
Red Slow
Blue Fast

The model never saw combinations such as:

  • red and fast;
  • blue and slow.

Later, the researchers showed the model a video of a mass moving quickly and changed its color to red. This created a conflict between the two clues.

The model had to decide whether to predict:

  • slow motion because the mass was red, or
  • fast motion because the mass had already been moving quickly.

Measuring the model’s answer

The researchers generated many videos and measured the motion of the mass in the completed videos. They checked whether the mass moved at a slow or fast frequency.

They also changed the color gradually from red to blue. This helped them see how strongly color influenced the model’s prediction.

Editing the model’s internal information

The model does not directly create the final video all at once. It processes information through many layers, somewhat like a long series of decision-making steps.

The researchers made an activation edit. This means they changed some of the numbers inside the model while it was working.

A simple analogy is changing directions in a GPS system while it is planning a route. The researchers tried to replace the model’s “red means slow” internal signal with the signal from a matching fast-motion example.

They tested:

  • different layers of the model;
  • different strengths of the edit;
  • different amounts of observed motion;
  • different training methods.

They also used a mathematical tool called PCA, or principal component analysis. In everyday language, PCA helps find the few most important directions in a large collection of numbers. The researchers found that only about four important internal directions were often enough to control the motion.

Finally, they tested the idea in a much larger, pretrained video model called Wan, which has about 1.3 billion parameters. Parameters are the adjustable numbers that store what a model has learned.

4. What did they find?

The model often uses a shortcut

The model can use color as a shortcut instead of carefully following the observed motion.

For example, when shown a fast-moving red mass, it may generate slow motion because it learned that red usually goes with slow movement.

This is called shortcut learning. It is similar to a student answering a question using a clue that usually works, even when the clue is misleading in a new situation.

The correct physical answer remains inside the model

Even when the model generates the wrong motion, the researchers could edit its internal state and make it generate the correct motion.

This shows that the model had not completely failed to learn fast motion. Instead, the correct answer was present but was not controlling the final video.

The researchers describe this as:

  • information is still present;
  • but it may no longer have enough control over the output.

A small internal edit can change the video

The researchers found that a low-dimensional edit, based on simple physical information such as:

  • the object’s position;
  • its velocity;
  • whether the desired motion should be fast or slow;

could successfully change the generated motion.

In many tests, the edited videos recovered almost all of the desired motion. The paper reports held-out recovery values around 0.94 to 0.99 for several experiments, meaning the edited result was very close to the target behavior.

The model eventually “commits” to a future

Edits worked well in earlier layers of the model. But after a certain point, the same edit no longer changed the generated motion.

The researchers call this point closure or commitment.

A useful analogy is writing with a pencil and then using an eraser:

  • early in the process, the writing can still be changed easily;
  • later, the drawing may be covered with ink, making the original plan harder to replace.

Importantly, the correct motion signal had not completely disappeared after this point. It was still detectable inside the model, but it no longer strongly affected the final video.

Stronger downstream edits can sometimes restore the correct motion

Although ordinary edits stopped working after commitment, a stronger edit to the model’s later attention mechanism could sometimes bring back the correct motion.

However, increasing the editing strength too much caused problems. The generated motion could overshoot and become unnatural.

This shows that controlling the model is not simply a matter of “turning up the volume.” The edit must be applied in the right place and with the right strength.

More evidence helps preserve the correct answer

The correct physical motion stayed writable deeper into the model when:

  • the model could see more frames of the object’s previous movement;
  • the model had stronger evidence about the motion;
  • the model learned motion before learning the misleading color relationship.

For example, the researchers trained one model first on videos without color differences, so it had to learn motion itself. This motion-first training produced better physical behavior and delayed the point where the model committed to the color-based shortcut.

Early writability predicts future learning

The researchers compared mistakes that training later corrected with mistakes that remained.

At an early stage of training, the errors that were later corrected could be changed at more network layers. Errors that remained wrong were writable at fewer layers.

This suggests that causal writability can predict whether a mistake is likely to be fixed by more training.

The findings also appeared in a larger video model

The researchers reproduced the main ideas in the pretrained 1.3-billion-parameter Wan video model.

This suggests that the result is not limited to one small, specially designed experiment. However, the authors stress that more research is needed to know whether the same behavior appears in realistic videos with people, animals, vehicles, and complicated scenes.

5. Why are these findings important?

The paper changes how we should think about mistakes made by video-generation models.

A wrong video does not always mean that the model lacks the correct knowledge. Sometimes the model may know several possible futures but choose one using an unreliable shortcut.

The research suggests that improving AI may require two separate goals:

  1. Teaching the model the correct information.
  2. Making sure the correct information can still control the final answer.

This could help researchers build video models that:

  • follow physics more accurately;
  • avoid relying too heavily on appearance or other shortcuts;
  • respond better to corrections;
  • reveal why particular mistakes happen;
  • use internal controls to repair incorrect predictions.

The idea may also help with AI safety and reliability. If researchers can find where a model’s correct knowledge becomes difficult to access, they may be able to intervene before the model fully commits to a bad prediction.

Simple conclusion

The main message of the paper is:

A video model can choose the wrong future even when it already contains the right physical answer.

The correct answer may remain hidden inside the model, but it becomes harder to use as the model processes the video. By editing the model at the right place, researchers can sometimes restore the physically correct motion.

This gives scientists a new way to study AI mistakes: not only asking “Does the model know the answer?”, but also asking “Can the answer still control what the model does?”

View Paper Prompt  View All Prompts 

Knowledge Gaps

Knowledge gaps, limitations, and open questions

  • Generality beyond synthetic, highly controlled dynamics remains unestablished. The main evidence comes from rendered spring, pendulum, and free-fall videos with simple trajectories, limited appearance variation, and known physical parameters; it is unclear whether causal writability occurs in natural videos involving articulated bodies, fluids, collisions, occlusion, camera motion, or multiple interacting objects.
  • The phenomenon has not been tested systematically across diverse pretrained video models. The paper reports one pretrained Wan 1.3B model, but does not establish how writability varies across architectures, parameter scales, tokenizations, VAE designs, training objectives, or diffusion/flow-matching samplers.
  • It remains unknown whether naturally occurring shortcuts in broad web-scale training exhibit the same closure boundary. The experiments deliberately create a color–motion correlation, whereas real-world shortcuts may involve semantic context, camera viewpoint, object identity, texture, or dataset-specific metadata.
  • The causal meaning of “closure” is specific to the tested intervention. A write becomes ineffective only for the particular observed-frame edit, injection sites, edit strength, decoding procedure, and checkpoints studied; the paper does not determine whether closure reflects a general loss of controllability or merely the failure of one intervention direction.
  • The mechanism underlying the sharp depth boundary is not fully explained. The study localizes the effect to observed-frame key/value information in self-attention, but does not identify the precise computation that converts this information into an irreversible future-token commitment.
  • The relationship between closure and irreversible commitment is unresolved. The paper shows that a stronger downstream intervention can sometimes restore motion, but does not establish whether commitment is genuinely irreversible under an unrestricted set of interventions or only difficult to reverse with the tested edit family.
  • The causes of across-seed behavioral differences remain unclear. Independently trained models show substantially different natural physics-following rates while sharing transferable edits, but the paper does not identify which parameter differences, optimization events, attention heads, or circuit-level changes produce these divergent downstream uses.
  • The shared cross-model edit space is only partially characterized. Transfer is demonstrated among three similarly trained Spring models and directions are aligned using a fitted scale and four-dimensional orthogonal map; the dimensionality, stability, and semantic meaning of this space across architectures, datasets, and physical systems remain unknown.
  • The four-dimensional edit representation may not be intrinsically physical. PCA coordinates are learned from paired aligned–conflict activation differences, so the reported low-dimensional structure could reflect the chosen intervention construction or correlated dataset variables rather than a model-independent physical representation.
  • The donor-free controller’s extrapolation limits are not tested. Controllers are evaluated within the studied slow/fast families and known boundary-state ranges; it is unknown whether they can generate effective edits for unseen frequencies, amplitudes, phases, forces, masses, or qualitatively different target dynamics.
  • The controller depends on access to physical state variables that may be unavailable in realistic settings. The proposed edits use boundary position, velocity, target direction, or gravity, but the paper does not address how these quantities would be estimated robustly from noisy, occluded, or imperfectly tracked observations.
  • The intervention can alter appearance as well as motion. Slow-to-fast edits shift future color toward blue, indicating entanglement between physical and appearance variables; the paper does not quantify whether motion can be corrected while preserving object identity, texture, lighting, and other non-target attributes.
  • The trade-off between physical correction and visual fidelity is insufficiently evaluated. Validity and fitted frequency are the primary success criteria, leaving open whether edited videos preserve sharpness, temporal consistency, spatial structure, object identity, and plausible rendering outside the tracked mass trajectory.
  • The effect of intervention strength is not characterized quantitatively across models and layers. Overshooting is demonstrated for one attention-head intervention in one run, but the safe gain range, nonlinear response curve, and relationship between gain and different physical targets are not systematically mapped.
  • The temporal allocation of effective writes remains poorly understood. The paper observes that late or all-call flow-matching interventions work better than early-only interventions, but does not determine which denoising times are causally critical or why the same internal signal has different effects at different sampling steps.
  • The interaction between network depth and denoising time is not disentangled. Closure is analyzed mainly as a depthwise phenomenon, while intervention effectiveness also depends on flow-matching time; a joint depth-by-time causal map is still missing.
  • The predictive relationship between early writability and later correction is correlational. Higher writability at 5K distinguishes errors later corrected from persistent errors, but the paper does not show that increasing writability through a targeted intervention causes later training to correct those examples.
  • The mechanisms governing reopening of writability during training are unknown. Writability can close and later reopen across checkpoints, but the paper does not identify whether reopening results from parameter drift, changes in attention routing, altered representations, optimization noise, or evolving competition between cues.
  • The effect of training order is confounded with adaptation history. Neutral-first and direct adaptation differ in the sequence of data and optimization exposure; further controls are needed to separate the effect of cue-independent pretraining from differences in total training trajectory, checkpoint initialization, and effective sample weighting.
  • The role of training-data scale and diversity is not examined. The synthetic experiments use 2,048 videos with two colors and two frequency bands; it remains unknown whether more varied data eliminates the shortcut, produces more robust physical representations, or creates additional competing predictive rules.
  • The paper does not test interventions under distribution shifts beyond the constructed color–motion conflicts. Open cases include novel colors, continuous rather than bimodal frequencies, unseen amplitudes, altered backgrounds, changes in camera geometry, and combinations of multiple simultaneous conflicts.
  • The scope of the “physical alternative” is limited. The experiments mainly redirect the model toward a known, training-supported physical future; they do not establish whether causal writability supports genuinely novel counterfactuals that were not represented during training.
  • The relationship between writable information and model knowledge is not fully resolved. Detectability of an aligned-minus-conflict activation direction after closure shows that a particular signal survives, but it does not establish that the model retains a complete, correctly parameterized physical simulation or can use the information without externally supplied edits.
  • Alternative explanations for activation-patching results remain possible. Because interventions use matched-run differences and low-dimensional projections, the observed effects could partly arise from distributional displacement, feature superposition, or subspace artifacts rather than a uniquely localized physical circuit.
  • Component restoration does not fully establish necessity and sufficiency. Restoring conflict K+V removes recovery and restoring Q has little effect, but the experiments do not test comprehensive combinations, head-level ablations, or independently learned replacements sufficient to prove that the identified attention pathway is the sole causal route.
  • The evaluator may bias the conclusions toward simple harmonic motion. Frequency fitting, validity thresholds, and band-based labels are tailored to oscillatory trajectories and may miss other forms of physical error, partial correction, phase drift, amplitude errors, or multimodal futures.
  • The statistical evidence is limited for some mechanistic claims. Several key findings rely on three Spring seeds, one pretrained model, selected strict-failure banks, or 48 trajectories; broader replication and uncertainty estimates are needed for claims about mechanism, transfer, and universal closure behavior.
  • The persistence of causal writability under noisy or imperfect interventions is unknown. The edits are computed from precisely matched aligned–conflict pairs or fitted controllers, whereas realistic model editing may involve estimation error, approximate alignment, noisy state measurements, and imperfect target specifications.
  • It is unclear whether causal writability can be converted into a practical inference-time correction method. The paper demonstrates diagnostic and manually designed edits, but does not evaluate automated detection of shortcut failures, automatic target selection, gain calibration, computational overhead, or robustness across arbitrary inputs.
  • The safety and failure modes of downstream amplification are unexplored. Excessive gain can produce off-family outputs, but the paper does not characterize whether amplification can cause instability, hallucinated objects, identity swaps, severe artifacts, or failures that evade the validity detector.
  • The connection to controllability and observability is conceptual rather than formalized. The paper invokes control-theoretic language but does not define a state-space model, controllability metric, or theorem linking the measured writability profile to standard notions of observability, reachability, or controllability.
  • The persistence of writability across longer horizons is unknown. All primary experiments use a fixed 64-frame prediction horizon; it remains unclear whether alternative futures remain writable for longer rollouts or whether closure becomes more pronounced as prediction depth increases.
  • Multi-object and compositional scenarios are not addressed. It is unknown whether an edit targeting one object’s dynamics can be applied without disrupting other objects, interactions, contact constraints, scene context, or global conservation laws.
  • The paper does not determine whether correcting the internal physical write improves natural generation without intervention. The intervention restores a desired future for selected cases, but it remains open whether training or architectural changes that preserve writability translate into broadly improved unedited video quality and physical consistency.
View Paper Prompt  View All Prompts 

Practical Applications

Immediate Applications

  • Diagnostic benchmark for shortcut learning in video models — AI research and software.
    • missing physical knowledge,
    • available but unused knowledge, and
    • physical information that has become causally inaccessible.

Dependencies: The current evidence is strongest for synthetic oscillators, pendulums, and free-fall scenes. Real-world benchmarks would require reliable object tracking, physical-state annotations, and carefully matched counterfactuals.

  • Layerwise “commitment” auditing for generative video systems — software engineering and model evaluation. The depth scan over network blocks can become an audit tool that identifies where a model commits to a selected future and can no longer be redirected by a fixed intervention. This could be integrated into model validation pipelines for video diffusion or flow-matching systems, producing a writability profile across layers and denoising steps.

Such a tool could flag models that: - commit too early to appearance-based predictions, - lose access to physically plausible futures, - or exhibit unstable behavior under moderate latent interventions.

Dependencies: The edit must be calibrated for the architecture and intervention site; closure is not necessarily a universal property of all edits or checkpoints.

  • Counterfactual physical steering during video generation — creative tools and simulation software.
    • “make the object move faster,”
    • “continue with the observed velocity,”
    • “increase gravity,” or
    • “change the oscillation direction.”

The edit would be injected into condition tokens during selected network blocks and denoising calls, while leaving the model weights unchanged.

Dependencies: The controller currently assumes known or estimable physical variables and a compatible learned representation. It must be calibrated separately for each model, motion family, and target state.

  • Training-data and checkpoint debugging — industrial ML development.
    • adding counterfactual color–motion combinations,
    • increasing the amount of observed motion,
    • reweighting persistent failure cases,
    • or changing the training curriculum.

Dependencies: The predictive relationship was demonstrated in a controlled setting and should be validated across datasets, architectures, and error types before being used as a production selection criterion.

  • Curriculum design that separates appearance from dynamics — robotics, simulation, and embodied AI.
    • robot manipulation and navigation,
    • autonomous driving simulation,
    • physical forecasting,
    • and interactive digital environments.

The immediate implementation is to randomize object color, texture, lighting, and context while preserving motion labels and then gradually restore realistic correlations.

Dependencies: The benefit may depend on the order, duration, and strength of the neutral phase. It is not yet established that the same curriculum improves downstream control or task success in physical robots.

  • Data augmentation for reducing visual shortcuts — computer vision and healthcare imaging.
    • varying object texture while preserving motion in robotics,
    • varying weather or vehicle color while preserving traffic dynamics,
    • varying scanner appearance while preserving clinical anatomy in medical video,
    • and varying classroom or presenter appearance while preserving instructional content.

Dependencies: Counterfactual transformations must preserve the relevant causal structure. Incorrect augmentation could remove meaningful signals or create physically implausible examples.

  • Mechanistic interpretability education and research tooling — academia.
    • a known simulator,
    • matched input pairs,
    • low-rank activation edits,
    • decoded-output metrics,
    • and layerwise intervention scans.

Dependencies: Reproducibility depends on access to the released code, exact checkpoints, deterministic evaluators, and intervention-compatible model implementations.

  • Quality assurance for synthetic media — media production and policy compliance. A decoded-video physics test can be added to generation systems that produce instructional, scientific, or engineering content. For example, a system could automatically test whether generated objects obey specified motion bands, gravity, collision, or conservation constraints, and reject or revise outputs that fail.

Dependencies: The paper measures relatively simple motion. General-purpose physical validation requires broader detectors, object identity tracking, collision reasoning, and uncertainty estimates.

Long-Term Applications

  • Physics-aware world models for robotics and autonomous systems — robotics and embodied AI. Causal writability could support world models that not only represent alternative physical futures but also retain the ability to select them during planning. A future system might:
    1. infer the current physical state,
    2. generate several candidate futures,
    3. steer internal representations toward a desired action or constraint,
    4. and evaluate the resulting decoded trajectory before execution.

Potential uses include robot grasping, contact-rich manipulation, drone flight, and autonomous driving under unusual conditions.

Dependencies: The paper does not establish reliable control in complex 3D environments, under partial observability, or with multi-object interactions. Safety-critical deployment would require closed-loop validation, uncertainty estimation, and guarantees against intervention-induced artifacts.

  • Interactive video simulators with physical-state controls — games, digital twins, and training environments.
    • engineering design,
    • emergency-response training,
    • flight and vehicle simulation,
    • education,
    • and game development.

Dependencies: Low-dimensional edits may work only within the model’s learned manifold. Extrapolation beyond training-supported physical modes could produce visually plausible but physically invalid results.

  • Physics-constrained video generation for science and engineering — scientific visualization. The controller-based approach could evolve into a system for generating counterfactual experiments, such as changing gravity, damping, material properties, or initial conditions. Rather than sampling unconstrained videos, the generator would be conditioned on a structured physical state and checked against a simulator or differentiable physical evaluator.

Dependencies: The current study uses simple, known dynamics and does not demonstrate quantitative accuracy for unseen laws, high-dimensional systems, fluid dynamics, or chaotic processes. Hybrid neural–numerical systems would likely be required.

  • Automated model repair without retraining — production AI and safety engineering. If a model retains a physically correct solution but fails to use it, activation-space controllers could serve as lightweight adapters or runtime patches. A deployed system might maintain a library of calibrated corrections for known failure modes and apply them only when a monitor detects a shortcut or implausible trajectory.

Dependencies: Runtime edits can overshoot, as shown by excessive attention gain. A practical repair system needs reliable failure detection, gain calibration, rollback mechanisms, and guarantees that corrections do not damage unrelated behaviors.

  • Cross-model transfer of interpretable physical controls — foundation-model ecosystems. The observed transfer of physical edits across independently trained models suggests a possible standard interface for physical representations. Different video models could share adapters describing concepts such as velocity, phase, acceleration, or gravity, even when their natural generations differ.

This could lead to: - portable control adapters, - model-agnostic physical evaluation suites, - shared latent-control APIs, - and transfer learning between simulation and generation models.

Dependencies: The demonstrated transfer occurs among closely related models and controlled tasks. Larger architectural differences, tokenization changes, and different training domains may require learned alignment maps and could invalidate direct transfer.

  • Causal auditing of high-stakes generative systems — policy, governance, and compliance. The distinction between information being represented and information being causally usable could inform audits of AI systems used in healthcare, finance, law, or public policy. An auditor could ask not only whether a model encodes a relevant factor, but whether that factor can still change the output after competing cues have been processed.

Examples include testing whether: - a medical video model uses motion rather than demographic proxies, - a financial forecasting model uses market dynamics rather than correlated metadata, - or a risk model can respond to a genuine causal intervention rather than a superficial feature.

Dependencies: Activation interventions are not automatically valid causal explanations; the paper itself notes risks of misleading subspace edits. Regulatory use would require domain-specific causal models, independent behavioral tests, and reproducible intervention protocols.

  • Adaptive denoising schedules that preserve controllability — generative-model architecture. Since effective control depends on both network depth and the timing of flow-matching interventions, future generators could allocate computation adaptively. Physical constraints might be injected at denoising stages where the model remains writable, while later stages refine appearance without overriding the selected dynamics.

Dependencies: The optimal intervention window may vary by prompt, object, motion type, and model architecture. It remains unclear whether such scheduling can scale to long videos and complex scenes without reducing visual quality.

  • Personalized and interactive learning tools — education. A controllable video model could generate visual demonstrations of mechanics from specified initial conditions, allowing students to alter mass, velocity, damping, or gravity and observe predicted consequences. The system could also diagnose whether a student’s requested scenario is being represented and correctly rendered by the model.

Dependencies: Educational deployment requires numerical fidelity, transparent uncertainty, and protection against plausible-looking but incorrect demonstrations. The present results support a research prototype, not a validated physics tutor.

  • Daily-life assistance through physically grounded prediction — consumer AI. In the longer term, household or wearable assistants could use video models to predict object motion, identify imminent falls or collisions, and simulate alternative actions. Causal steering could allow the system to compare “what will happen naturally” with “what would happen if the object were stopped, redirected, or accelerated.”

Dependencies: Real environments contain occlusion, multiple interacting objects, nonrigid materials, sensor noise, and ambiguous intent. Safe use would require conservative uncertainty handling and integration with explicit perception and physics modules rather than relying on latent edits alone.

View Paper Prompt  View All Prompts 

Glossary

  • Activation editing: Modifying internal neural-network activations to alter a model’s output. “A low-dimensional internal edit predicted from simple physical variables restores the correct fast motion.”
  • Activation patching: Replacing activations from one model execution with activations from another to investigate causal mechanisms. “Activation patching probes internal mechanisms”
  • AdamW: An optimization algorithm combining Adam adaptive updates with decoupled weight decay. “Optimization & AdamW with PyTorch ConstantLR”
  • Attention head: A subcomponent of multi-head attention that independently computes attention-weighted representations. “In Run C, we amplify only the observed-frame V of one of the nine heads”
  • Bidirectional Transformer: A Transformer architecture in which tokens can attend to information on both sides of their position. “30 bidirectional DiT blocks”
  • Causal intervention: A deliberate modification of an internal variable or process to test whether it causes a change in an outcome. “an intervention can give it control.”
  • Causal writability: The capacity of an internal intervention to change a model’s generated output. “Causal writability is the ability of an internal edit to change motion in decoded video.”
  • Checkpoint: A saved state of a model’s parameters during or after training. “At an early checkpoint, those errors are writable at more network depths than errors that persist.”
  • Closure: The depthwise point at which an intervention no longer changes the model’s output. “Closure marks commitment for that write.”
  • Concept activation vector: A vector in activation space representing a human-interpretable concept and usable for steering model behavior. “concept activation vectors steer VideoMAE's plausibility judgments”
  • Controllability: In control theory, the ability to drive a system into desired states through suitable inputs. “This parallels the control-theoretic distinction between observability and controllability”
  • Counterfactual matching: Constructing paired inputs that differ in a controlled variable while holding other relevant variables fixed. “Our mechanistic analysis uses controlled synthetic dynamics, where exact counterfactual matching and quantitative decoded-video evaluation are possible.”
  • Cross-entropy loss: A loss function measuring the discrepancy between predicted probability distributions and target outcomes. “increasing their cross-entropy loss.”
  • Cross-model transfer: Applying an intervention or learned representation from one model to another model. “Cross-model steering transfers interventions”
  • Decoded video: A video reconstructed from a model’s latent representation into pixel space. “We measure motion directly in fully decoded videos.”
  • Denoising: The iterative removal of noise or uncertainty from a latent representation during generative inference. “early-denoising cross-attention write”
  • DiT: A diffusion Transformer architecture that uses Transformer blocks as the denoising network. “a pretrained 1.3B video DiT”
  • Diffusion model: A generative model that learns to produce data by reversing a gradual noise-injection process. “Video Diffusion Models”
  • Flow matching: A generative-modeling method that learns a vector field transporting noise distributions toward data distributions. “a 488M-parameter latent flow-matching Transformer”
  • Frozen model/component: A model or submodule whose parameters are held fixed during a particular procedure. “a frozen Wan2.1 VAE”
  • Generative world model: A model that predicts how an environment or scene evolves over time, often for simulation or planning. “Generative world models learn from video to predict how scenes will evolve”
  • Latent representation: An internal, typically lower-dimensional representation of data used by a generative model. “A frozen Wan2.1 VAE maps each 129-frame video to a 33×16×16 latent grid”
  • Manifold steering: Guiding model behavior by modifying representations along directions associated with a learned representation manifold. “Manifold steering links geometry to behavior, including video physics”
  • Mechanistic analysis: An investigation of the internal computational components and causal pathways responsible for a model’s behavior. “Our mechanistic analysis uses controlled synthetic dynamics”
  • Multi-head attention: An attention mechanism that computes several attention operations in parallel using separate learned projections. “Head-level responsibility differs across runs”
  • Observability: In control theory, the ability to infer a system’s internal state from its outputs or measurements. “a physical solution may remain detectable after it loses control over generation.”
  • Orthogonal map: A linear transformation that preserves inner products, lengths, and angles. “A scale and four-dimensional orthogonal map fitted on separate trajectories transfers controller-predicted edits”
  • Predictive shortcut: A feature or correlation that enables prediction without representing the intended underlying causal relationship. “The model can follow color and generate slow motion---a predictive shortcut.”
  • Predictive underspecification: A situation in which training data are compatible with multiple predictive rules that differ under novel or conflicting inputs. “This is predictive underspecification”
  • Residual state: The activation carried through a residual connection and updated by adding a block’s computed transformation. “the pre-block residual”
  • Self-attention: An attention mechanism in which elements of a sequence use other elements of the same sequence to update their representations. “self-attention carries motion information from observed frames into future-frame states”
  • Shortcut learning: Reliance on easy but potentially non-causal correlations instead of robust, intended features. “Shortcut learning and predictive underspecification.”
  • Subspace intervention: An intervention that modifies activations only within a selected lower-dimensional subspace. “subspace interventions can mislead causal attribution”
  • Token: A discrete computational unit representing part of an input sequence or latent representation. “Only observed-frame tokens are changed.”
  • Variational autoencoder (VAE): A probabilistic encoder–decoder model that maps data to a continuous latent distribution and reconstructs it. “a frozen Wan2.1 VAE”
  • Vector field: A function assigning a vector, such as a velocity or update direction, to every point in a space. “Flow matching for Generative Modeling”
  • Writability boundary: A depth or computational boundary beyond which a specified intervention ceases to affect the output. “At fixed strength, we find a sharp depth boundary over a narrow range of layers”
  • Zero-shot transfer: Applying a learned capability or intervention to a new setting without additional task-specific training. “corrective edits transfer between them.”
View Paper Prompt  View All Prompts 

Open Problems

We haven't generated a list of open problems mentioned in this paper yet.

Generate Now

Tweets

Sign up for free to view the 4 tweets with 125 likes about this paper.

Sign Up for Free