CineCtrl: Fine-Grained Generative Cinematic Control

• CineCtrl is a video cinematic editing framework that enables fine-grained generative control over both camera trajectory and photographic parameters.
• Its architecture features a novel decoupled cross-attention mechanism to separate and accurately manage motion and photographic inputs.
• Evaluation shows CineCtrl outperforms text-driven and stitching baselines by delivering precise bokeh and exposure control with superior scene consistency.

CineCtrl is a video cinematic editing framework that provides fine control over professional camera parameters, including bokeh and shutter speed, within a generative video model, while preserving scene consistency across time (Sun et al., 17 Nov 2025). In the broader literature, the term also denotes a control-centric research direction in which cinematography is treated as a structured optimization or generation problem over camera trajectory, framing, lens behavior, and affective style rather than as text-only prompt following; earlier systems formulated closely related goals in virtual cinematography, engine-based previsualization, robotic filming, and controllable video synthesis (Wu et al., 2023).

1. Emergence and research context

CineCtrl emerged against a background in which most automatic cinematography systems controlled either camera extrinsics alone or a small set of predefined shot types. Earlier virtual cinematography work framed the core problem as automatic camera trajectory control that creates immersion through actor–camera synchronization in three aspects: frame aesthetics, spatial action, and emotional status in a 3D virtual stage (Wu et al., 2023). Parallel systems exposed adjacent production workflows: OpenKinoAI targeted post-production of ultra high definition live-performance footage through detection, tracking, reframing into cinematographic rushes, multiclip editing, and annotation (Ronfard et al., 2020), while Cine-AI focused on procedural generation of in-game cutscenes in the style of human directors through a Unity-based timeline and storyboard interface plus runtime cinematography automation (Evin et al., 2022).

A second lineage came from robotics and autonomous camera control. CineMPC introduced nonlinear MPC for autonomous cinematography with joint control of camera extrinsics and intrinsics, including focal length, focus distance, and aperture, making depth of field, zoom, and composition explicit optimization targets rather than post hoc effects (Pueyo et al., 2021). Its later fully autonomous formulation incorporated RGB-D perception, multi-target tracking, and a modular ROS implementation for drone cinematography (Pueyo et al., 2024). CineTransfer then used a single example video to extract composition and depth-of-field style and supply those as time-varying references to CineMPC, thereby moving from user-authored objectives toward style-conditioned control (Pueyo et al., 2023).

A third lineage addressed generative and interactive video. Image Conductor separated camera motion from object motion through camera LoRA weights, object LoRA weights, and camera-free guidance in a conditional video diffusion model (Li et al., 2024). CinePreGen combined a 3D engine, storyboard-level camera authoring, and diffusion-based rendering for previsualization, explicitly centering camera placement and cinematic camera movement in the workflow (Chen et al., 2024). ChatCam extended control to conversational language, using CineGPT, an Anchor Determinator, and an LLM agent to translate text into camera trajectories in radiance-field scenes (Liu et al., 2024).

Within this context, CineCtrl is distinguished by shifting the control problem from camera trajectory alone to generative photographic control. Its stated target is scene-consistent video cinematic editing with fine-grained, independent control over professional camera parameters such as depth of field, focal length, exposure, and color temperature, alongside camera trajectory (Sun et al., 17 Nov 2025).

2. Control space and formal representation

CineCtrl formulates cinematic editing as a conditional transformation from a source video $V_s$ and a photographic control signal $P$ to an edited target video $V_t$: $V_t = \mathcal{G}(V_s, P),$ where $\mathcal{G}$ is a generative video model that preserves scene semantics while altering photographic appearance and, when specified, camera trajectory (Sun et al., 17 Nov 2025).

The control space is explicitly partitioned into two streams. The first is camera trajectory, represented by

$$P_{\text{traj}} \in \mathbb{R}^{\tau \times 3 \times 4},$$

a per-frame sequence of relative camera extrinsic matrices between source and target. The second is the photographic parameter stream,

$$P_{\text{pho}} \in \mathbb{R}^{\tau \times 5}, \qquad P_{\text{pho}} = [K, d_f, f, \mathcal{S}, \mathcal{T}],$$

where $K$ is bokeh blur intensity, $d_f$ is refocused disparity, $f$ is focal length, $P$0 is shutter speed, and $P$1 is color temperature (Sun et al., 17 Nov 2025). The normalization is asymmetric: $P$2, whereas $P$3.

This partition is significant because earlier control systems often entangled trajectory with other visual changes or treated zoom as part of camera motion. CineCtrl instead models photographic parameters as a distinct control manifold. In practical terms, this makes bokeh intensity separable from focal-plane placement, exposure separable from scene geometry, and tone separable from viewpoint. This suggests a shift from “where the camera is” toward “how the camera sees.”

The photographic parameters are grounded in explicit physical or quasi-physical formulations. For depth of field, CineCtrl uses a blur radius driven by

$P$4

where $P$5 is disparity and $P$6 is the focal-plane disparity (Sun et al., 17 Nov 2025). For zoom, focal length controls field of view through

$P$7

For exposure, a relative shutter-speed multiplier is defined as

$P$8

and for color temperature the source temperature $P$9 K is mapped to a target temperature $V_t$0 by piecewise interpolation over $V_t$1 (Sun et al., 17 Nov 2025).

This explicit control vocabulary is congruent with the camera pillar of Stable Cinemetrics, which formalizes professional video generation around intrinsics, extrinsics, trajectory, and creative intent, and identifies shot size, angle, depth of field, aperture, shutter speed, ISO, and camera movement as separate control nodes (Chatterjee et al., 30 Sep 2025). A plausible implication is that CineCtrl’s parameterization can be read as an operational subset of that broader taxonomy, with especially strong emphasis on the photographic branch.

3. Model architecture and decoupled conditioning

CineCtrl is built on Wan 2.1, a Diffusion Transformer based text-to-video model operating in a 3D VAE latent space and trained with a Flow Matching-style objective (Sun et al., 17 Nov 2025). It converts this backbone into a video-to-video editor. The source and target videos are encoded as

$V_t$2

then patchified into token sequences

$V_t$3

with

$V_t$4

The model concatenates them into

$V_t$5

The architectural novelty is the Camera-Decoupled Cross-Attention mechanism. Rather than injecting all camera-related controls through a single pathway, CineCtrl computes a shared query

$V_t$6

from the video features $V_t$7, then forms separate keys and values for trajectory and photographic streams: $V_t$8

$V_t$9

Two parallel cross-attention branches are then evaluated: $V_t = \mathcal{G}(V_s, P),$0 and fused by

$V_t = \mathcal{G}(V_s, P),$1

The output projection $V_t = \mathcal{G}(V_s, P),$2 is zero-initialized so that the model initially behaves like the frozen Wan 2.1 prior (Sun et al., 17 Nov 2025).

The purpose of this decoupling is explicitly to disentangle camera motion from photographic inputs. The paper reports that naïve element-wise addition of control features caused entanglement, artifacts, and degraded motion smoothness, whereas decoupling improved bokeh control accuracy and scene consistency (Sun et al., 17 Nov 2025). Architecturally, this places CineCtrl in dialogue with Image Conductor, which also separated motion factors, but did so through distinct LoRA branches for camera and object motion plus camera-free guidance in a diffusion model (Li et al., 2024). The difference is that Image Conductor disentangled camera and object motion, whereas CineCtrl disentangles trajectory and photographic control.

4. Data generation and training regime

CineCtrl addresses a shortage of training data for professional camera parameters through a combined synthetic and real-world pipeline (Sun et al., 17 Nov 2025). The synthetic portion starts from the ReCamMaster multi-camera dataset and applies simulated photographic effects to produce source–target pairs with known trajectory differences and known photographic parameter differences. The real-world portion contributes 32k clips produced through a three-stage pipeline: shot detection with PySceneDetect, video filtering, and post-processing.

The filtering stage is highly structured. Clips shorter than 81 frames are removed, overly dark clips are filtered by average grayscale intensity, and clips dominated by close facial shots are removed using MediaPipe face detection and a bounding-box area ratio criterion. A further information-content filter measures displacement by extracting ORB features, matching them with a BF matcher plus ratio test, fitting an affine transform with RANSAC, and aggregating transform magnitudes over sliding windows of different sizes; clips with low scores at both small and large windows are discarded (Sun et al., 17 Nov 2025). Post-processing then estimates camera poses with MegaSaM and captions clips with Qwen2.5-VL before re-applying the same physical simulation pipeline to create paired edits.

Training combines approximately 170k synthetic videos with 32k real clips. Source–target pairs are drawn from five sets: original, bokeh-modified, zoom-modified, exposure-modified, and color-temperature-modified. A subtle but important supervision strategy is used for disentanglement: when source and target are identical, CineCtrl sets $V_t = \mathcal{G}(V_s, P),$3 but randomizes $V_t = \mathcal{G}(V_s, P),$4, forcing the model to learn that focal-plane placement should have no visual effect when blur intensity is zero (Sun et al., 17 Nov 2025). The model is trained for 160k steps on 8× A800 GPUs with batch size 8. Only $V_t = \mathcal{G}(V_s, P),$5, the projector before camera conditioning, and the decoupled cross-attention layers are fine-tuned; $V_t = \mathcal{G}(V_s, P),$6 is initialized from ReCamMaster and the rest of Wan 2.1 remains frozen.

This training design belongs to a wider pattern in cinematography research. Virtual camera generation systems have often relied on synthetic or artist-authored datasets: the actor-synchronized camera controller of “Automatic Camera Trajectory Control with Enhanced Immersion for Virtual Cinematography” used 13,066,689 synthetic samples for its aesthetic adjustor and 25,230 artist-designed or augmented five-second sequences for trajectory synthesis (Wu et al., 2023). In controllable video generation, Image Conductor similarly used a trajectory-oriented curation pipeline based on WebVid, RealEstate10K, cut detection, motion filtering, and CoTracker-derived trajectories (Li et al., 2024). CineCtrl’s contribution is to extend this logic from camera motion data to photographic supervision.

5. Interfaces, control modalities, and system integration

CineCtrl’s primary interaction model is numeric and time-varying. Users specify per-frame or keyframed curves for $V_t = \mathcal{G}(V_s, P),$7, $V_t = \mathcal{G}(V_s, P),$8, $V_t = \mathcal{G}(V_s, P),$9, $\mathcal{G}$0, $\mathcal{G}$1, together with relative camera extrinsics $\mathcal{G}$2 (Sun et al., 17 Nov 2025). This makes the framework closer to a virtual professional camera than to a prompt-only generator. Typical editing scenarios include increasing bokeh intensity while sweeping the focal plane, performing zooms by increasing focal length, brightening or darkening scenes through $\mathcal{G}$3, and shifting warm or cool mood through $\mathcal{G}$4. The framework also supports concurrent changes in trajectory and photographic parameters.

The broader literature shows several adjacent interface paradigms for CineCtrl-like systems. Cine-AI used a Unity Timeline integration with shot markers, design-time simulation, and a storyboard interface for per-shot inspection and locking (Evin et al., 2022). CinePreGen introduced a camera and storyboard interface with dynamic control from global to local camera adjustments, built around CineSpace and engine-powered diffusion (Chen et al., 2024). ChatCam replaced graphical controls with conversation, using GPT-4, CineGPT, and an Anchor Determinator to decompose instructions into atomic trajectories and anchors (Liu et al., 2024). OpenKinoAI, in a different production context, organized reframing, multiclip editing, and annotation through a web client backed by server-side analysis and rush generation (Ronfard et al., 2020).

These systems do not define CineCtrl itself, but they delineate the design space into which it fits. A plausible implication is that CineCtrl’s current parameter-curve interface could be embedded in timeline-based, storyboard-based, engine-based, or conversational front ends without changing its underlying control representation. In that sense, CineCtrl is less a fixed user interface than a control substrate for photographic editing.

6. Evaluation, limitations, and research significance

CineCtrl is evaluated against text-driven baselines and a stitching baseline using both automatic metrics and a user study (Sun et al., 17 Nov 2025). The automatic evaluation includes CorrCoef for photographic control accuracy, LPIPS and CLIP-V for scene consistency, CLIP-F for temporal coherence, and VBench metrics covering aesthetic quality, imaging quality, temporal flickering, motion smoothness, subject consistency, and background consistency. CorrCoef is measured by generating pseudo ground truth through the physical simulator and comparing the realized effect strength in model outputs to that pseudo target.

The reported qualitative pattern is consistent across effects. CineCtrl outperforms both zero-shot and fine-tuned text baselines on bokeh, zoom, exposure, and color temperature control, while the stitching baseline suffers visible artifacts from cascading multiple non-end-to-end modules (Sun et al., 17 Nov 2025). In the user study, involving 100+ participants and 30 trials, preference for CineCtrl reached 80.91% against the unfine-tuned text baseline, 97.65% against the fine-tuned text baseline, and 96.15% against the stitching baseline. Ablations further show that removing decoupled cross-attention reduces bokeh CorrCoef and degrades motion smoothness and subject/background consistency, and that removing the real dataset especially harms focal-plane control through $\mathcal{G}$5 (Sun et al., 17 Nov 2025).

The framework’s limits are also explicit. Extreme parameter values can produce artifacts; training clips are 81–100 frames, so very long or rapidly changing videos remain challenging; shutter speed is used for exposure rather than explicit motion-blur control; and geometric complexity beyond the training distribution can still limit the accuracy of $\mathcal{G}$6 (Sun et al., 17 Nov 2025). These limits align with a broader diagnosis from Stable Cinemetrics, which found that even strong contemporary video models exhibit significant gaps in professional control, especially for Camera-related controls and Events, with the Camera pillar remaining weak across models and becoming worse under joint “Director” prompts that specify all control dimensions simultaneously (Chatterjee et al., 30 Sep 2025).

Within the field, CineCtrl therefore marks a specific transition. Earlier controllable video systems concentrated on camera motion, object motion, or high-level shot planning; CineCtrl makes photographic camera parameters—bokeh, focal length, exposure, and color temperature—first-class editable variables inside a generative video model (Sun et al., 17 Nov 2025). This suggests a redefinition of cinematic control in generative video: from trajectory-conditioned synthesis toward a fuller camera model in which optics and exposure are optimized or authored alongside movement.