CineVision: AI Pre-visualization Storyboarding
- CineVision is an AI-driven storyboard pre-visualization tool that translates scripts into editable visual references for directors and cinematographers.
- It integrates script input with SQL-like metadata queries and a MovieNet database to rapidly generate cinematographically styled stills for scene planning.
- The system emphasizes real-time collaborative iteration and visual control over lighting, style, and character design, while currently limiting output resolution and camera-path details.
CineVision is an AI-driven pre-visualization and storyboard system for director-cinematographer collaboration during pre-production. It integrates script input, movie-database retrieval, and real-time visual regeneration so that a scene can be translated into editable storyboard stills with controllable lighting, director-style presets, character design, costume, and background. In the published system, the emphasis is not on explicit 3D camera-path optimization or animation, but on rapid iteration over cinematic still images that can function as shared visual references for shot selection, mood exploration, and communication across production departments (Wei et al., 27 Jul 2025).
1. Definition and production role
CineVision is designed specifically for pre-production tasks such as script breakdown, storyboarding, and look exploration. Its stated aim is to address the difficulty of turning a script’s abstract description into precise, shared visual ideas, the slow and skill-dependent nature of hand-drawn storyboards, the inconsistency of single-prompt diffusion workflows, and the repeated clarification loops between directors and cinematographers about lighting, style, and character appearance (Wei et al., 27 Jul 2025).
The system operates primarily as a pre-visualization and storyboard-creation environment. A director provides script text and optional structured constraints, CineVision retrieves matching shot groups from a preprocessed MovieNet database, and the selected references are then regenerated under new visual controls. The final storyboard stills are intended to guide camera placement, blocking, and practical lighting decisions in prep meetings, and to align costume and art departments with an agreed visual direction (Wei et al., 27 Jul 2025).
Its scope is therefore narrower than a full virtual production stack. CineVision generates still images only; it does not provide skeletal or facial animation, explicit camera paths, or lens metadata such as focal length, T-stop, or sensor crop. A plausible implication is that the system functions as a boundary object between script interpretation and later shot-list or on-set execution, rather than as a complete cinematographic automation layer (Wei et al., 27 Jul 2025).
2. Interaction model and storyboard workflow
The operational workflow begins with script input. A dialogue or script segment is pasted into the interface, and optional SQL-like constraints can be specified for attributes such as location type, time of day, and number or gender of characters. The backend then queries MovieNet and retrieves coherent shot groups, for example two-person dialogue compositions in a corridor at night (Wei et al., 27 Jul 2025).
The selected shot group is pinned to a real-time preview board, after which only the pinned frames are regenerated when the user presses “Reshot.” CineVision preserves the original references alongside the regenerated outputs, enabling side-by-side comparison during iterative discussion. The system is explicitly organized around co-present collaboration: the director and cinematographer work together at one interface, inspect the same references, adjust the same controls, and negotiate mood, angle, lighting, costume, and style in real time (Wei et al., 27 Jul 2025).
The interface includes a script editor, an SQL-like metadata panel, a reference-shot gallery, a real-time preview board, an optional free-text prompt box, control panels for background, lighting, director style, character design, and costume, and system parameter controls such as guidance scale, randomness, and number of steps. The design prioritizes shared visual vocabulary and non-destructive iteration, since original frames remain visible while new variants accumulate (Wei et al., 27 Jul 2025).
Although CineVision does not expose formal camera-optimization controls, composition is indirectly determined by the retrieved reference frames and the system’s regeneration model. Shot groups are drawn from dialogue-oriented movie stills, including over-the-shoulder and alternating close-up patterns. The paper states that a preset library of about 180 dialogue framings covers more than 90% of cases observed in the formative study, which situates the system within conventional live-action dialogue staging rather than arbitrary visual composition (Wei et al., 27 Jul 2025).
3. System architecture and data-to-image pipeline
CineVision uses a browser-based frontend built with Gradio and a backend running on an Ubuntu server with an RTX 4090 (24GB) GPU. The backend combines a MovieNet-based film still database and metadata, SQL-style querying, CLIP-based ranking, and PyTorch-based fine-tuned Stable Diffusion 1.5 models (Wei et al., 27 Jul 2025).
The text-to-film retrieval stage does not perform deep narrative parsing via LLMs. Instead, it combines raw script text with structured user metadata and issues a SQL-like query over MovieNet scenes, following ideas from ScriptViz. The query filters by metadata such as location, time, and number of characters, and also applies a scene-complexity filter that excludes crowded shots with more than 6 visible characters. Candidate frames are ranked using CLIP visual-text similarity between stored setting tags and the script description, together with face recognizability metrics intended to avoid blurry or low-quality frames (Wei et al., 27 Jul 2025).
The generative stack is based entirely on Stable Diffusion 1.5. Two main fine-tuning stages are described. The first targets relighting and environment style: 1,000 MovieNet images of two-person dialogue scenes were curated under criteria including exactly two speaking characters in close or medium shots, no blur or exposure issues, diverse lighting, and balanced composition. ChatGPT-4o generated descriptions emphasizing lighting and style, after which Stable Diffusion 1.5 was fine-tuned for 5 epochs on a single A100 80GB and then adapted using the IC-Light method for relighting capacity and intra-scene lighting consistency (Wei et al., 27 Jul 2025).
The second fine-tuning stage targets character and costume fidelity. It uses 2,000 MovieNet portrait images selected for clear face and upper body, diverse costume eras and body types, diverse ages and emotional states, and good exposure and sharpness. ChatGPT-4o generated character- and costume-focused descriptions, and Stable Diffusion 1.5 was again fine-tuned for 5 epochs to improve response to portrait, facial, and clothing prompts (Wei et al., 27 Jul 2025).
Prompt control is organized hierarchically rather than through unconstrained free-form prompting. If categories are denoted as and their importance weights as , CineVision defines total conditioning weight as
$W_{\text{total} = \sum_{i=1}^{n} w_i \cdot x_i$
with first-tier categories such as environment, time of day, actor attributes, and director style assigned higher weights than second-tier details, and with facial detail and expression weighted more heavily than minor clothing color. This weighted prompt composition is intended to stabilize outputs by prioritizing structural scene cues before fine facial and wardrobe refinements (Wei et al., 27 Jul 2025).
4. Visual control model: lighting, style, character, costume, and environment
A defining feature of CineVision is its use of cinematography-oriented abstractions rather than low-level parameterization. Lighting control is presented through time-of-day presets such as Noon, Night, and Sunrise/Sunset; light-type controls such as Soft Light, Hard Light, and Key Light; and direction choices such as left or right light source. The relighting model, derived from IC-Light, is intended to preserve physically plausible behavior and smooth multi-source transitions while maintaining intra-shot consistency (Wei et al., 27 Jul 2025).
Style emulation is likewise exposed through high-level presets. The published system includes 10 director-style options, including Wes Anderson, Martin Scorsese, Stanley Kubrick, Ridley Scott, and Russo Brothers. These presets are not implemented as per-director fine-tuned models; instead, they are prompt-based templates constructed from a curated reference process in which a filmmaker-author selected representative films and shots, and GPT-4o was iteratively prompted to describe style in terms of lighting, color palette, and character styling. The resulting textual descriptions are inserted into the weighted prompt system (Wei et al., 27 Jul 2025).
Character design is handled through menu-driven attribute control over facial features, expressions, hair length, style, and color. Costume design provides clothing categories such as T-shirt, tank top, business attire, and dresses, together with style labels, fabric texture, and color. Environments are selected from a curated catalogue of more than 100 background types, including bedrooms, corridors, and beaches. The system aims to preserve identity, outfit, and environmental continuity across all shots in a scene unless intentionally modified (Wei et al., 27 Jul 2025).
CineVision’s camera and composition control remains indirect. The paper explicitly states that there is no rule-of-thirds optimization or explicit camera-path model. Instead, composition emerges from the retrieved film stills, the system’s dialogue-framing priors, and the model’s learned ability to preserve framing under regeneration. This suggests a workflow optimized for storyboard ideation rather than for explicit camera mechanics (Wei et al., 27 Jul 2025).
5. Empirical evaluation and collaboration outcomes
CineVision was evaluated in a 24-participant lab study comprising 12 female and 12 male participants with mean age approximately 25.4. The cohort included 6 professionals with more than 1 year in film work—3 directors and 3 cinematographers—and 18 amateurs who had created short films or videos. Participants were paired into director-cinematographer dyads across three groups: CineVision, DALL·E 3 as an AI baseline, and hand-drawn storyboards (Wei et al., 27 Jul 2025).
All groups completed the same 60-minute protocol: 10 minutes of training on the assigned tool and a 50-minute storyboard-creation task for a 7-frame scene consisting of an ambient opening shot and 6 dialogue beats based on a six-line emotionally charged dialogue between Ethan and Olivia. Measures included NASA-TLX, a User Experience Questionnaire, a dyadic collaboration score, and semi-structured interviews (Wei et al., 27 Jul 2025).
Several workload dimensions favored CineVision. For Physical Demand, Group A scored versus for DALL·E 3 and for hand drawing, with . For Temporal Demand, CineVision scored versus and , with 0. For Effort, the values were 1, 2, and 3, with 4. For Frustration, CineVision yielded 5 versus 6 and 7, with 8. Mental Demand and Performance did not show significant differences (Wei et al., 27 Jul 2025).
Usability measures also favored CineVision. On Usefulness, Group A scored 9 compared with $W_{\text{total} = \sum_{i=1}^{n} w_i \cdot x_i$0 for DALL·E 3 and $W_{\text{total} = \sum_{i=1}^{n} w_i \cdot x_i$1 for hand drawing, with $W_{\text{total} = \sum_{i=1}^{n} w_i \cdot x_i$2. On Ease of Use, CineVision obtained $W_{\text{total} = \sum_{i=1}^{n} w_i \cdot x_i$3 compared with $W_{\text{total} = \sum_{i=1}^{n} w_i \cdot x_i$4 and $W_{\text{total} = \sum_{i=1}^{n} w_i \cdot x_i$5, with $W_{\text{total} = \sum_{i=1}^{n} w_i \cdot x_i$6. The collaboration score was $W_{\text{total} = \sum_{i=1}^{n} w_i \cdot x_i$7 for CineVision, $W_{\text{total} = \sum_{i=1}^{n} w_i \cdot x_i$8 for DALL·E 3, and $W_{\text{total} = \sum_{i=1}^{n} w_i \cdot x_i$9 for hand drawing, with 0 (Wei et al., 27 Jul 2025).
Qualitative observations are consistent with these numerical results. CineVision dyads reportedly spent more time discussing lighting, mood, and style than fighting prompt syntax or clarifying sketch meaning. DALL·E 3 users spent substantial time dealing with unpredictable outputs and character inconsistency, while hand-drawing teams were constrained by drawing skill. A professional cinematographer in the CineVision group stated that the atmosphere, colors, and communication provided by the system were more accurate and saved discussion time, while a professional director emphasized the value of the system’s visual references for selecting shots and angles (Wei et al., 27 Jul 2025).
6. Position within the broader cinematic-vision research landscape
Within the specific 2025 system, CineVision denotes a storyboard and pre-visualization environment. In adjacent literature, however, the term also aligns with a broader class of film-aware AI systems that parse cinematic intent, synthesize or evaluate camera behavior, and generate visually coherent shot sequences. This suggests that CineVision can be understood both as a concrete platform and as part of a larger research trajectory toward cinematic vision systems (Wei et al., 27 Jul 2025).
LensCraft, for example, addresses prompt- or script-level camera control rather than still-image storyboard generation. It translates high-level cinematographic intent into full 6-DoF camera trajectories using a volume-aware representation built around a volume bounding box, an attention bounding box, subject motion, and standardized cinematographic descriptions. It supports prompt-only, keyframe-conditioned, and trajectory-conditioned inference, and reports a balanced dataset of 100,000 samples, approximately 3M frames, together with markedly lower computational cost than diffusion-based camera generators (Dehghanian et al., 1 Jun 2025). A plausible implication is that LensCraft provides the kind of trajectory engine that a still-image pre-visualization system like CineVision does not presently include.
VERTIGO extends this trajectory-centered line of work by introducing render-in-the-loop visual preference optimization. It renders candidate trajectories in Unity, evaluates them with a cinematically fine-tuned vision-LLM, and uses Direct Preference Optimization to improve framing and prompt adherence. Reported results include reducing Missing Rate from 0.387 in GenDoP to 0.008 while preserving geometric fidelity, and reducing the character off-screen rate from 38% to nearly 0% (Li et al., 2 Apr 2026). This addresses a misconception common in trajectory-generation research: geometrically plausible camera motion is not equivalent to visually desirable framing.
CineVerse focuses on another adjacent problem: consistent keyframe synthesis for cinematic scene composition. It generates multi-shot keyframes from a high-level scene description by first using an LLM to plan setting, character descriptions, and shot descriptions, then fine-tuning a text-to-image generator for joint multi-shot synthesis. The framework uses explicit shot-size descriptions and reports strong gains over multi-image baselines in shot alignment, scene alignment, and correct shot-count generation (Phung et al., 28 Apr 2025). Compared with CineVision, CineVerse is more directly concerned with structured multi-shot synthesis, whereas CineVision emphasizes human-in-the-loop retrieval, regeneration, and collaboration.
Other nearby work addresses the rendering and imaging substrate rather than storyboard interaction. CineScale targets 8K image generation and 4K video generation from existing diffusion backbones through self-cascade upscaling and frequency-aware attention fusion (Qiu et al., 21 Aug 2025); dynamic novel view synthesis work explores NeRF- and Gaussian-Splatting-based virtual re-shoots and smooth synthetic camera moves for cinematography (Azzarelli et al., 2024); and the aximorphic projection model generalizes real-time camera projection beyond linear perspective to rectilinear, fisheye, anamorphic, and mixed lens behaviors (Fober, 2021). Together, these systems indicate that the broader CineVision problem spans storyboard communication, camera planning, visual preference alignment, scene-consistent keyframe generation, neural rendering, and lens modeling.
7. Limitations and likely development directions
The published CineVision system has several explicit limitations. Output resolution is fixed at 1, and higher resolutions or very complex scenes can reduce interactivity. The system is tuned mainly for conventional live-action dialogue scenes, and quality degrades for large-scale or highly layered setups. Style control is limited to 10 director presets, and there is no user-defined style-learning or upload-your-own-style capability (Wei et al., 27 Jul 2025).
Functionally, CineVision remains a still-image system. It lacks skeletal animation, facial animation, explicit camera motion, and explicit shot metadata export for focal length or lensing decisions. Collaboration is organized around a director-cinematographer dyad rather than broader simultaneous multi-role participation, and evaluation was limited to a controlled lab study with short sessions and a single scene rather than longitudinal deployment in production pipelines (Wei et al., 27 Jul 2025).
The stated future directions include higher resolution, improved handling of complex scenes, user-defined style learning, multi-role collaboration features, export of shot-critical metadata, integration with tools such as Unreal, FBX, or CSV shot lists, and longer-term on-set studies. Related research suggests additional directions. LensCraft indicates one path toward real-time camera-path synthesis from text and keyframes (Dehghanian et al., 1 Jun 2025); VERTIGO indicates the value of director-in-the-loop visual preference post-training (Li et al., 2 Apr 2026); and CineVerse suggests a stronger LLM-based scene-planning layer for explicit shot-sequence generation (Phung et al., 28 Apr 2025). This suggests that a future CineVision system could evolve from interactive storyboard regeneration into a more integrated cinematic operating environment linking script understanding, shot planning, trajectory generation, rendering, and evaluative feedback.