Hy-Nomas

• Hy-Nomas is an 8-billion-parameter video diffusion world model designed for free-form interactive world generation, capable of preserving object identities, scene states, and viewpoint consistency across multi-turn interactions and camera movements.
• This model stands out for its ability to handle semantic actions, such as moving objects and camera movement, and maintaining visual and state consistency over prolonged interactions through a structured prompt enhancement mechanism.
• Hy-Nomas supports a variety of tasks, including generating complex scenarios, controlling camera trajectories, and ensuring state transitions remain coherent, making it effective for storytelling, animation, and interactive sim.

Hy-Nomas is an 8-billion-parameter video diffusion world model for free-form interactive world generation (Chen et al., 27 Oct 2026). It generates variable-length video segments in response to successive text instructions and continuous camera trajectories, while conditioning each segment on cached visual, textual, and camera history. The system is designed to preserve object identity, scene state, action consequences, and viewpoint consistency across multi-turn interactions rather than generating isolated fixed-duration clips.

1. Problem formulation and interaction model

Hy-Nomas models interactive generation as a closed-loop process. The system first generates a video segment, receives a new user instruction, and then generates a subsequent segment conditioned on the instruction and prior interaction history. The world is not reset between turns.

Supported interventions include semantic actions such as picking up or moving objects, scene changes, continuous camera motion, and mixed sequences that alternate semantic commands with camera control. The target behavior requires more than temporal ordering of frames: the model must represent state transitions, preserve consequences of previous actions, avoid repeating completed commands, and maintain visual consistency under viewpoint changes.

At inference time, a vision-language Prompt Enhancer (PE) receives the current instruction, previous user requests, PE-generated prompts, previously generated video segments, and optional camera-control inputs. It produces a structured caption and a duration for the next segment. The diffusion transformer then conditions on the structured text, optional camera trajectory, and cached visual and textual history. The interaction is represented by

$$y_t \sim p_{\theta}\!\left( \cdot \mid S,u_t,H_{t-1},V_{<t},m_t \right),$$

where $y_t$ is the structured caption for turn $t$, $u_t$ is the current instruction, $H_{t-1}$ is the dialogue history, $V_{<t}$ is the previously generated video history, and $m_t$ specifies the rewriting mode.

The first turn establishes the environment, subjects, style, and viewpoint. Later text turns produce incremental descriptions containing description_change and contained_actions. Camera turns produce scene-description changes while supplying the camera trajectory separately to the diffusion model.

2. Structured data and temporal supervision

Hy-Nomas is trained on heterogeneous real and virtual videos containing human and object motion, physical interactions, object manipulation, camera movement, long-range temporal dependencies, and diverse viewpoints, styles, and environments. The data pipeline splits raw videos into five-minute chunks, detects shot boundaries using OmniShotCut, transcodes clips, filters them by duration, aspect ratio, resolution, motion, clarity, aesthetics, and transition quality, and removes duplicates using visual embeddings, hierarchical clustering, and SemDeDup.

Selected videos receive camera-trajectory estimation, hierarchical taxonomy labels, and structured chronological captions. Caption sampling is adapted between training stages to emphasize weaknesses identified during evaluation.

Each structured annotation contains video-level visual style, camera perspective, a shared subject list, persistent subject identifiers, static subject profiles, segment-level scene descriptions, camera-motion descriptions, timestamped actions, and visible action consequences. The first segment gives a full description; subsequent segments specify changes relative to the established world. Persistent identifiers such as subject_1 allow later actions to refer to the same subject without reintroducing a new textual identity.

Actions are represented separately from scene descriptions. Their intervals are chronological, non-overlapping, and constrained to segment boundaries. Camera-motion sentences are enclosed in <camera> ... </camera>. When reliable camera trajectories are available, these sentences are removed and explicit trajectory conditioning is used; otherwise, the tagged sentences remain as textual supervision.

Caption generation is itself multi-turn. The first vision-language-model pass establishes the environment and subject profiles, later passes describe changes using preceding visual segments, and a final text-only pass validates JSON structure, timestamps, subject references, consistency, and English–Chinese alignment. The paper names Gemma 4 and Qwen3.5-122B-A10B for large-scale captioning and Kimi K3 for higher-quality annealing annotations.

3. Architecture and generative formulation

Hy-Nomas adapts HunyuanVideo 1.5 into an autoregressive interactive model. Its principal components are a causal DCAE-V video autoencoder, a frozen Qwen2.5-VL-7B text encoder, a timestep-aware text-token refiner, 54 dual-stream MMDiT blocks, segment-wise and action-aware causal attention, explicit camera cross-attention, a 3D RoPE positional scheme, a causal KV cache, and a 3D VAE decoder. A PixelUMM-derived decoder is used for deployment optimization.

Component	Configuration
Video autoencoder	DCAE-V
Compression	$32\times$ spatial and $4\times$ temporal
Latent channels	32
Text encoder	Frozen Qwen2.5-VL-7B
Transformer	54 dual-stream MMDiT blocks
Hidden and MLP widths	2,048 and 8,192
Attention	16 heads, 128 dimensions per head
Camera representation	64-dimensional grouped extrinsics
Prediction target	32-channel latent flow velocity

For an RGB video $X\in\mathbb{R}^{B\times 3\times F\times H\times W}$, the causal encoder produces

$y_t$0

with

$y_t$1

Every latent position becomes a token and is projected into the 2,048-dimensional transformer space. The high spatial and temporal compression reduces the number of tokens required for long interactive histories.

Hy-Nomas uses linear flow matching. Given clean latent $y_t$2, Gaussian noise $y_t$3, and $y_t$4,

$y_t$5

The model predicts the velocity $y_t$6 from the noisy latent and conditioning information. The corresponding flow-matching objective is

$y_t$7

where $y_t$8 includes text, camera, and history conditioning. Generation begins from Gaussian noise and integrates toward the clean latent. The original system uses 100 Euler steps per segment; diffusion distillation reduces inference to four network function evaluations.

4. Causal attention, memory, and camera control

Hy-Nomas partitions video and text tokens into aligned temporal segments. A query in segment $y_t$9 can attend to the current and preceding segments but not to later segments. Attention is therefore bidirectional within a segment and causal across segments.

Segment-wise causality alone does not prevent a video token from attending to later actions in the same segment, nor does it prevent subsequent segments from repeatedly attending to old action instructions. Hy-Nomas introduces action-aware causal attention by dividing each segment’s text into a descriptive head and action-specific tails. The descriptive head carries persistent scene context, including layout, subject identity, visual style, camera perspective, and persistent object descriptions. Action tails are accessible only within their originating temporal windows.

For a video query token in action window $t$0, the accessible text keys are restricted to the descriptive head and the corresponding action tail:

$t$1

Later segments retain access to the descriptive head of an earlier segment but not to its action tails. The visual consequences of actions remain in the generated video history after the textual instruction is no longer visible to the model. This design is intended to reduce action-order errors and cross-segment action repetition.

After generation, visual and textual key-value representations are inserted into a causal cache. Conceptually,

$t$2

Deployment uses bounded-cache inference with sliding-window attention. Older context may be offloaded or discarded according to the window policy, reducing memory requirements while potentially introducing long-rollout differences.

Camera control is represented through relative camera-to-world transformations. Given camera-to-world poses $t$3,

$t$4

Translations are normalized by a video-level scale defined as the 99th percentile of translation magnitudes. Groups of four $t$5 transformations are flattened into 64-dimensional vectors and projected into camera tokens aligned with VAE latent frames. Each MMDiT block injects camera information through

$t$6

where visual tokens query temporally aligned camera tokens.

Keyboard inputs are converted into trajectories: W/S control forward and backward movement, A/D lateral movement, and mouse or arrow inputs control yaw and pitch. Speed, scale, and duration can be changed without retraining. Camera conditioning is randomly disabled during training. When enabled, textual camera-motion descriptions are removed; when disabled, they remain in the caption. Camera-conditioned and non-camera-conditioned examples are mixed at a 1:1 ratio during camera continual pre-training.

5. Training curriculum and systems engineering

The training procedure consists of five principal stages.

1. Latent-space adaptation: Hy-Nomas adapts HunyuanVideo 1.5 to DCAE-V. The first 20K steps update input and output layers while freezing intermediate DiT blocks; the next 60K steps train the full model.
2. Continual pre-training: Segment-causal training establishes causal generation, action-aware training introduces localized action timing, and camera-conditioned training adds continuous trajectory control. The action-aware stage increases context length from 180K to 360K tokens.
3. High-quality annealing: Higher-quality, often single-subject videos are used with denser captions. The maximum text length increases from 768 to 1,280 tokens, and reliable ViPE camera trajectories receive explicit supervision.
4. Segment forcing: The model is trained on its own partially denoised histories rather than only clean ground-truth histories. For segment $t$7, a context latent is constructed as

$t$8

The imperfect context is appended to the KV cache, approximating the conditions encountered during autoregressive inference. Gradients are accumulated over the complete sequence before the optimizer update.

1. Diffusion distillation: A causal version of Phased Consistency Models provides few-step initialization. Quality refinement uses hybrid forcing, causal Decoupled Distribution Matching Distillation, an adversarial hinge objective, and a flow-matching-trained fake-score model. At inference, generated segments are perturbed with flow noise at $t$9 before entering the cache.

The training system supports packed sequences of up to 360K tokens using Ulysses sequence parallelism with $u_t$0, data parallelism, FSDP2 hybrid sharding, activation checkpointing, CPU activation offloading, Flex-Flash-Attention/MagiAttention, FLOP-balanced sequence packing, Distributed Muon, AdamW, and asynchronous checkpointing.

Workload balancing accounts for both valid token count $u_t$1 and the number of unmasked query-key pairs $u_t$2:

$u_t$3

This is intended to estimate attention cost more accurately than token count alone under heterogeneous causal masks.

For deployment on an NVIDIA RTX 4090, the paper combines INT8 model weights, INT8 cached keys and values with floating-point scales, BF16 active-attention computation, sliding-window attention, CPU offloading, sparse attention after the first two denoising steps, a PixelDiT decoder derived from PixelUMM, and FP8 one-step decoder distillation. For a 100-second video generated in seven segments at $u_t$4 and 24 FPS, the reported peak GPU memory usage is 20.53 GiB.

6. Evaluation, adaptation, and limitations

Hy-Nomas-Bench contains 280 cases across four benchmarks: 60 multi-turn free-form text cases, 100 camera-navigation cases, 60 interleaved text-and-camera cases, and 60 spatial-memory cases. The tests include first-, second-, and third-person viewpoints, third-person observer views, photorealistic, game-like, and animated environments, real and virtual scenes, and multi-turn sequences longer than 60 seconds.

Three trained annotators evaluate pairwise outputs for prompt following, physical plausibility, camera control, and memory consistency. Results are reported as win, tie, and loss rates, with net win rate defined as win percentage minus loss percentage. Baselines include Genie 3, LingBot-World v2, and Happy Oyster.

Against Happy Oyster Directing on multi-turn text interaction, Hy-Nomas obtains a 65.0% win rate for prompt following and a 45.0% win rate for physical plausibility. Against LingBot-World v2, it obtains 75.0% and 68.3%, respectively. On interleaved text and camera control against the adapted LingBot-World v2, it obtains 75.0% for prompt following, 73.3% for physical plausibility, and 26.7% for camera control.

Spatial-memory results are more mixed. Against Genie 3, Hy-Nomas wins 13.3% of cases, ties 56.7%, and loses 30.0%. Against Happy Oyster, it wins 30.0%, ties 46.7%, and loses 23.3%. Against LingBot-World v2, it wins 58.3% and loses 1.7%. The reported results indicate stronger performance in text interaction and physical plausibility than in long-term spatial consistency. The authors associate Genie 3’s advantage with grounding in Street View data and suggest that more videos containing looped camera trajectories could improve Hy-Nomas.

The system supports LoRA adaptation. The recommended procedure is to train LoRA adapters on the base autoregressive model, merge them into the distilled model, and deploy the merged distilled model without repeating full distillation. The paper suggests that approximately 100 task-specific samples may suffice for lightweight customization, although actual data requirements depend on the task.

A cinematic LoRA uses rank 256, alpha 512, 500 iterations, and learning rate $u_t$5. It achieves 35% wins versus 24% losses for prompt following and 46% wins versus 13% losses for physical plausibility on Bench-1, but camera control regresses to 19% wins versus 37% losses. A robotics LoRA is trained on 128 dual-arm task videos from RealSource with rank 512 and approximately 1,500 epochs. It improves the plausibility of out-of-distribution dual-arm interactions, although executable robot-action prediction remains unresolved.

Hy-Nomas does not provide an explicit persistent 3D world state. Its memory consists primarily of cached visual, textual, and camera representations. Consequently, spatial layouts and object states can drift, especially across long rollouts. Error accumulation can produce identity changes, geometry changes, implausible motion, and scene breakdown. Bounded caches and deployment approximations trade memory and speed for possible losses in long-range consistency.

The model is a video world model rather than a guaranteed physical simulator. Temporal ordering and visual plausibility do not establish physical-law correctness or executable action outcomes. The Prompt Enhancer is another dependency: errors in interpreting instructions, preserving history, selecting durations, or producing structured captions propagate into the diffusion model.

The benchmark is human-evaluated and limited to 280 cases. It does not directly measure exact geometric trajectory error, physical-law correctness, executable robot-policy success, long-horizon state estimation, standardized visual fidelity, or real-time latency under all resolutions and rollout lengths. The paper releases model weights, source code, Hy-Nomas-Bench, benchmark prompts, PE-enhanced prompts, and a lightweight LoRA adaptation workflow.

Hy-Nomas is therefore best characterized as a multi-turn, free-form, camera-controllable visual simulator built by combining structured chronological supervision, action-aware causal attention, persistent cached history, continuous camera conditioning, a vision-language Prompt Enhancer, model-generated-history training, and few-step diffusion distillation. Its strongest reported capabilities are free-form text interaction, physical plausibility, and unified text–camera control; its principal unresolved limitation is long-term spatial and state consistency.