AirScape: 6 DoF Aerial World Model

• AirScape is a generative world model for 6 DoF aerial agents that predicts future observation sequences from first-person visual inputs and motion commands.
• It combines an 11,000-clip aerial video–intention dataset, an adapted video-diffusion backbone, and a dual-phase training schedule to enforce motion controllability and physical consistency.
• Quantitative metrics such as improved FID, FVD, and IAR highlight its potential for autonomous drone navigation, planning, and virtual cinematography.

AirScape is a generative world model for six-degree-of-freedom aerial agents that predicts future observation sequences from current egocentric visual input and motion intentions. It is presented as the first world model designed for 6 DoF aerial agents, with the stated objective of enabling robots to predict the outcomes of their own motion intentions in three-dimensional space. The system combines an 11 000-clip aerial video–intention dataset, a pretrained video-diffusion backbone adapted for 6 DoF control, and a two-phase training schedule intended to enforce both motion controllability and physical spatio-temporal consistency (Zhao et al., 10 Jul 2025).

1. Research problem and conceptual scope

The central problem addressed by AirScape is the following: given a current first-person visual observation and a 6 DoF motion intention, predict the future stream of images that an aerial agent would observe. In the formulation provided, this is the problem of answering “what will I see if I execute this command in three-dimensional space?” for an autonomous drone (Zhao et al., 10 Jul 2025).

Two difficulties are identified as fundamental. The first is high-dimensional, rapid appearance change. Aerial flight combines forward and backward motion, lateral and vertical translation, and rotations about all three axes, often together with a changing camera-gimbal angle. The described consequences are large parallax shifts, narrow field-of-view effects, and swift changes in perspective, all of which increase the difficulty of frame-by-frame prediction. The second is motion controllability. A generic video generator may produce plausible dynamic scenes from rich textual prompts, but a world model must remain tightly aligned with the agent’s specific intentions, such as moving forward until a building fills the frame or rotating $90^\circ$ to the left.

This distinction is important for embodied intelligence. AirScape is not framed merely as a conditional video synthesis system; it is framed as a model that must jointly support large spatial imagination in egocentric video and tight coupling between commanded motion and predicted observation. This suggests that its intended use is not only visual generation but also planning and counterfactual reasoning in aerial robotics.

2. Dataset construction and intention representation

AirScape is trained and evaluated on an 11 000-clip dataset $\mathcal{D}$ of video–intention pairs (Zhao et al., 10 Jul 2025). The source material comes from three publicly available UAV datasets: UrbanVideo-Bench, NAT2021, and WebUAV-3M. Each raw video was segmented into 129-frame clips, after which clips with very little motion (“static”) or abrupt, unrealistic jumps were filtered out.

The annotation pipeline combines automated proposal and human refinement. An LMM, described as a LLM of Motion, was prompted in a simple chain-of-thought fashion to describe the action, its stopping condition, and any task-level vernacular. Examples in the description include action phrases such as “move forward,” stopping conditions such as “until close to the building,” and task-level descriptions such as “track the white car ahead.” More than 1 000 human-hours were then spent correcting mistakes, including wrong action types, vague wording, and incorrect task definitions.

Although the dataset uses high-level text $p$ for human-friendly intent, the instantaneous motion intention at time $t$ is also represented as a 6D vector,

$$a_t=\bigl(x_t,\;y_t,\;z_t,\;\phi_t,\;\theta_t,\;\psi_t\bigr),$$

where $(x,y,z)$ denotes desired translation in world coordinates and $(\phi,\theta,\psi)$ denotes roll, pitch, and yaw rotations. This dual representation establishes a bridge between natural-language instruction and continuous control.

The dataset is described as covering more than 1 000 hours of annotated intention work and spanning 10+ environment types, including industrial parks, residential zones, suburbs, coastal areas, urban centers, and day, dusk, and night lighting. Action categories include pure translation, pure rotation, gimbal adjustments, and compound sequences. A plausible implication is that the dataset is intended to expose the model to both kinematic diversity and environmental variation rather than to a single narrowly defined flight regime.

3. Architectural design

AirScape builds on the pretrained video-diffusion backbone CogVideoX-i2v-5B and adapts it for 6 DoF control (Zhao et al., 10 Jul 2025). At each time step $t$, the model takes a current RGB observation $o_t\in\mathbb{R}^{C\times H\times W}$, a target textual intention $p$ or an embedded 6D representation $\mathcal{D}$0, and produces a predicted future clip $\mathcal{D}$1.

Internally, the model maintains a latent state $\mathcal{D}$2, with the update and decoding equations

$\mathcal{D}$3

In this formulation, $\mathcal{D}$4 is a dynamics module built from stacked Transformer blocks with cross-attention to the intention embedding and 3D convolutional self-attention on the latent state. The decoder $\mathcal{D}$5 is a U-Net-style decoder using 3D convolutions and attention to map the latent representation back into image space. In each cross-attention layer, the textual embedding of $\mathcal{D}$6, or a continuous vector representation of $\mathcal{D}$7, is injected.

The stated rationale is that this architecture allows simultaneous learning of spatio-temporal features through 3D convolutions and instruction alignment through cross-attention. In effect, AirScape is organized as a controllable latent dynamics model embedded within a diffusion-based video generation framework. This suggests a design choice aimed at reconciling high-capacity visual synthesis with explicit conditioning on aerial motion intent.

4. Two-phase training schedule

Training proceeds in two phases intended to establish, first, intention controllability and, second, physical consistency (Zhao et al., 10 Jul 2025).

In Phase 1, supervised fine-tuning is performed on the real dataset $\mathcal{D}$8. Writing $\mathcal{D}$9 for the first frame of $p$0, the optimization objective is the video reconstruction loss

$p$1

where $p$2 denotes the model output given observation $p$3 and intention $p$4. The stated function of this stage is to teach the model to follow simple instructions.

In Phase 2, spatio-temporal fine-tuning is driven by a self-play loop that constructs a synthetic dataset $p$5. For each real clip $p$6, an LMM proposes a set of paraphrases $p$7 of the same intent. For each paraphrase $p$8, the model rolls out $p$9 candidate videos $t$0 using different noise seeds. These candidates are then rejection-sampled by asking the LMM, acting as a critic, to rate them on four criteria: intention alignment, spatial consistency, temporal continuity, and projective geometry. The highest-rated pair $t$1 is retained in $t$2.

Once $t$3 is large enough, the model is fine-tuned again with

$t$4

where $t$5 penalizes deviation in predicted 6D pose increments relative to the intended $t$6, and $t$7 penalizes physically impossible spatial or temporal artifacts.

The training schedule encodes a specific division of labor. The first stage establishes conditional obedience to motion instructions; the second uses synthetic rollouts and critic-based filtering to favor outputs that satisfy geometric and temporal constraints. This suggests a hybrid supervision strategy in which direct reconstruction and model-generated self-improvement are both used to shape the learned world model.

5. Predictive performance and controllability

AirScape is reported to generate coherent future clips for unseen observations $t$8 and intentions $t$9, and the paper provides quantitative evaluation on held-out aerial sequences (Zhao et al., 10 Jul 2025). The reported metrics are Frame-wise FID, FVD for temporal coherence, and IAR (Intention Alignment Rate, measured by human raters).

Metric	AirScape	Baselines
FID	$$a_t=\bigl(x_t,\;y_t,\;z_t,\;\phi_t,\;\theta_t,\;\psi_t\bigr),$$0	$$a_t=\bigl(x_t,\;y_t,\;z_t,\;\phi_t,\;\theta_t,\;\psi_t\bigr),$$1–$$a_t=\bigl(x_t,\;y_t,\;z_t,\;\phi_t,\;\theta_t,\;\psi_t\bigr),$$2
FVD	$$a_t=\bigl(x_t,\;y_t,\;z_t,\;\phi_t,\;\theta_t,\;\psi_t\bigr),$$3	$$a_t=\bigl(x_t,\;y_t,\;z_t,\;\phi_t,\;\theta_t,\;\psi_t\bigr),$$4–$$a_t=\bigl(x_t,\;y_t,\;z_t,\;\phi_t,\;\theta_t,\;\psi_t\bigr),$$5
IAR	$$a_t=\bigl(x_t,\;y_t,\;z_t,\;\phi_t,\;\theta_t,\;\psi_t\bigr),$$6	$$a_t=\bigl(x_t,\;y_t,\;z_t,\;\phi_t,\;\theta_t,\;\psi_t\bigr),$$7–$$a_t=\bigl(x_t,\;y_t,\;z_t,\;\phi_t,\;\theta_t,\;\psi_t\bigr),$$8

The paper states that these averages are computed over translation, rotation, and compound actions. The interpretation given is that AirScape more accurately predicts what a drone would actually see given a 6 DoF command.

Qualitative capabilities are described in relation to specific maneuvers. For rotation maneuvers such as rotating $$a_t=\bigl(x_t,\;y_t,\;z_t,\;\phi_t,\;\theta_t,\;\psi_t\bigr),$$9 about yaw, the model is described as smoothly rotating the entire scene without twisting buildings or letting roads float. For forward and backward translation, objects are described as growing or shrinking with correct perspective. For compound actions such as moving up, rotating right, and tilting the gimbal down, the model is described as producing correct parallax shifts and consistent relative motion of foreground and background.

A recurring conceptual point is that motion controllability is not reducible to generic visual plausibility. The contrast drawn in the problem formulation implies a common misconception: a video model that produces realistic aerial footage is not necessarily a usable world model. In AirScape, usefulness depends on whether the predicted observation sequence remains aligned with the commanded 6 DoF motion.

6. Physical consistency, applications, and prospective extensions

AirScape is explicitly designed to satisfy physical spatio-temporal constraints in addition to command following (Zhao et al., 10 Jul 2025). The mechanism offered for this property is the combination of supervised fine-tuning and LMM-guided self-play. Candidate videos that violate geometry or temporal continuity are filtered out by the critic during self-play, and the subsequent loss includes both motion and physics terms. The paper states that physical consistency emerges because candidates that fail these constraints are rejected.

The stated application areas are autonomous drone navigation and planning, simulation environments for reinforcement learning, and virtual cinematography. In navigation and planning, the intended role is to allow an aerial agent to imagine outcomes of candidate maneuvers before execution. In reinforcement learning, the model is positioned as a source of realistic first-person visual streams conditioned on agent actions. In virtual cinematography, it can be used to design camera trajectories in 3D scenes from high-level intents.

Several future directions are identified. These include integrating AirScape with a real drone in closed-loop control for online re-planning, extending the world model to multi-sensor inputs such as depth and IMU and to multi-view outputs, scaling the self-play loop with stronger critics or adversarial discriminators, and exploring hierarchical intention representations ranging from text prompts to continuous control signals for end-to-end task planning.

Taken together, these directions indicate that AirScape is framed less as a terminal application than as an intermediate foundation for aerial embodied intelligence. This suggests a broader research trajectory in which generative world models are expected to mediate among perception, control, and planning under explicit physical and geometric constraints.