A Text-Native Interface for Generative Video Authoring

Abstract: Everyone can write their stories in freeform text format -- it's something we all learn in school. Yet storytelling via video requires one to learn specialized and complicated tools. In this paper, we introduce Doki, a text-native interface for generative video authoring, aligning video creation with the natural process of text writing. In Doki, writing text is the primary interaction: within a single document, users define assets, structure scenes, create shots, refine edits, and add audio. We articulate the design principles of this text-first approach and demonstrate Doki's capabilities through a series of examples. To evaluate its real-world use, we conducted a week-long deployment study with participants of varying expertise in video authoring. This work contributes a fundamental shift in generative video interfaces, demonstrating a powerful and accessible new way to craft visual stories.

• The paper introduces Doki, a text-native interface that consolidates video authoring into a single document, reducing tool switching and creative fragmentation.
• It employs hierarchical structuring with parameterized definitions using @Mentions and #Hashtags, along with integrated AI agents, to ensure narrative consistency and streamline production.
• Empirical evaluation shows high usability and rapid content prototyping, demonstrating Doki’s potential to empower both novices and expert users in generative video creation.

Doki: A Text-Native Interface for Generative Video Authoring

Motivation and Context

The authors address a central challenge in generative video authoring: the fragmentation of creative workflows imposed by traditional and contemporary tools. Despite the proliferation of text-to-video models such as Veo, Sora, Runway, and others, current systems primarily generate individual short clips and rely on multi-pane non-linear editors for assembly, which introduces cognitive overhead and impedes fluent narrative construction. The paper analyzes workflows observed among contemporary creators and identifies persistent problems: excessive tool/context switching, prompt-centric workflows that detract from narrative development, and difficulties in maintaining consistency across shots and assets.

Doki is introduced as an interface that centralizes video authoring within a single text-native representation that leverages structured text as a substrate for compositional video production, aiming to unify and simplify workflows, enhance narrative coherence, and lower barriers to entry. The design philosophy is grounded in four principles: centering authoring on freeform text, consolidating representation, enforcing consistency via parameterization, and simplifying interaction.

Interface Paradigm and Core Design

Doki’s interface departs from the prevailing “bento box” paradigm (multiple, disjointed asset/prompt/visual/audio panes) and consolidates all authoring steps into a canonical text-native document serving as both the semantic narrative and executable structure for generative video (Figure 1).

Figure 1: Comparison between “Bento Box” interfaces and Doki’s text-native representation, showing unified authoring via document-centric interaction.

Within this document, users can author reusable definitions (characters, scenes, styles), insert shots and sequences inline, define and propagate parameterized styles via mentions and hashtags, and invoke both sidebar and inline AI agents for agentic editing. The document view directly reflects the logical structure of the video: document $\rightarrow$ video, paragraphs $\rightarrow$ sequences, sentences $\rightarrow$ shots.

Two representative workflows are demonstrated: one driven by manual asset definition and narrative writing, and another emphasizing agentic interaction, draft generation, and iterative refinement with AI agents (Figure 2).

Figure 2: Workflows illustrating asset-first and agent-driven authoring via Doki’s interface.

Structured Representations, Parametrization, and Context Handling

Doki structures the authoring process hierarchically—a document maps to a video, each paragraph to a sequence, and sentences to shots. Inline shot previews are woven into text and serve as generation triggers; paragraphs containing multiple shots allow context inheritance and continuity, ensuring strong cross-shot consistency without the need for verbose prompts or repetitive context specification (Figure 3, Figure 4).

Figure 3: Inline shot creation with immediate visual preview and subsequent clip generation.

Figure 4: Contextual inheritance in multi-shot paragraphs for preserving continuity and consistency.

For parameterization, Doki's definitions system allows users to specify reusable assets via {@Mentions} (characters, scenes, props, reference frames) and {#Hashtags} (styles, camera movements, mood, spatial arrangements), all directly in text. Visual definitions (generated or user-uploaded images) can augment textual parametric references, propagating appearance and style to future generations (Figure 5, Figure 6).

Figure 5: Creation of definitions via slash commands, with optional visual preview.

Figure 6: Inline generation and selection of shot and definition variants.

Global and scoped definitions are supported: the special {#all} definition applies a style/content globally, while heading-based scoping ensures local overrides, enabling hierarchical organization (Figure 7).

Figure 7: Application of global and scoped definitions for document-wide or section-specific parameterization.

AI Agent Integration and Generative Pipeline

Doki embeds two forms of agent assistance: a sidebar conversational agent and an inline agent for granular edits. The inline agent supports immediate operations such as “Enhance” (making descriptions more vivid), “Create Definition” (extracting referenced assets), and “Custom Request” (user-instructed in-context edits) (Figure 8). All edits are visually tracked and auditable within the document.

Figure 8: Inline agent actions for enhancing, defining, and custom editing text selections.

The shot generation pipeline operates through staged transformation: user prompt entries are resolved into structured prompts (grounded in definitions and context), which are then rewritten for fluency and optimal model performance, before static images and video clips are produced. Contextual referencing from definitions and prior shots is utilized to maximize coherence (Figure 9).

Figure 9: Pipeline diagram showing reference resolution, prompt rewriting, and staged image/video generation.

Empirical Evaluation: Diary Study

A week-long mixed-methods diary study was conducted with 10 participants spanning varied expertise (filmmakers, animators, content creators, novices), yielding 46 video artifacts. Participants reported high engagement (average session $\approx$92 min), high utilization rates of shot generation, export, and AI agent features, and created videos across storytelling, instructional, advertising, music, and experimental genres (Figure 10).

Figure 10: Keyframes from participant-created videos demonstrating diverse supported genres.

System Usability Scale (SUS) scores averaged 81.2 (“Excellent”), with scores spanning 62.5-90.0. Participants highlighted fast transitions from ideation to draft, improved coherence via parameterization, and increased comprehension from document-centric views. Notably, users without prior filmmaking or generative AI experience used Doki to create video narratives that would have otherwise been inaccessible to them.

Expert users positioned Doki as complementary to traditional high-fidelity tools—utilizing Doki primarily for rapid ideation and storyboarding—while novices found it enabled creative expression beyond their technical capacity.

Interface analytics revealed high per-minute generation rates for images and videos, and robust utilization across export, agent, and variant features (Figure 11).

Figure 11: Usage analytics for session duration, generation cost, and per-minute activity rates.

Limitations and Design Trade-offs

Participants identified principal limitations: model output unpredictability, lack of precise frame compositional control, and temporal expressivity constraints inherent to linear document structure. Doki’s text-native paradigm simplifies the interface but restricts fine-grained spatial and concurrent temporal editing. Audio and timing features currently lack flexibility for complex synchronization or transitions.

Experts underscored that Doki—while providing high consistency across scenes and assets—cannot yet match animation industry standards; visual and compositional control remains insufficient for high-fidelity production. The design trade-off between compositional multi-view interfaces and minimalist single-representation interfaces is highlighted: Doki’s paradigm excels in accessibility and efficiency but sacrifices some task-specific precision.

Implications, Contributions, and Future Directions

Theoretically, Doki demonstrates that text can serve as a common ground for collaborative human–AI creative workflows, enhancing transparency, traceability, and revisionability compared to “direct input–output” generative pipelines. It reframes generative video authoring as document authoring, centralizing both narrative and production process.

Strong numerical results include rapid video production (commonly sub-15 minutes for 30–60s videos), robust usability outcomes (SUS median >80), and consistent participant engagement across expertise levels.

Practically, Doki’s findings reveal that approachability does not guarantee narrative quality; craft and domain knowledge remain essential. The study suggests future interfaces may incorporate narrative scaffolds, lightweight diagnostics, and document-embedded temporal primitives (event offsets, beat markers, transitions).

As generative models extend their capabilities (longer durations, stronger context, more control), the authors anticipate that progress in cross-shot memory, identity preservation, and story-aware control will yield greater advances in narrative authoring than mere increases in clip length. The paradigm shift towards text as the substrate for content creation interfaces is posited as essential as generative AI matures.

Additional Features and Infrastructure

The interface comprises a video preview player, settings panel for AI model selection and project management, cost monitoring for generation, and flexible export options (video, clip, JSON formats) (Figure 12).

Figure 12: Doki project utilities including player, settings, cost monitor, and export.

Doki is implemented with TypeScript/React, TipTap for rich text, Zustand for state, Node.js backend, FFmpeg for video, and supports a range of generative models including Gemini 2.5 Flash for text, Flux Kontext Pro for imagery, and Veo 3 Fast for video generation.

Conclusion

Doki introduces a text-native paradigm for generative video authoring, unifying and simplifying creative workflows, enhancing narrative coherence via parameterization, and lowering entry barriers. Empirical evidence demonstrates strong usability, accelerated idea-to-content flow, and increased sense of ownership—especially via agentic collaboration. Doki’s document-centric representation opens new directions for human–AI co-creation, emphasizing transparency and editability, while raising questions about balancing accessibility with temporal and spatial expressivity in generative media tools.

(2603.09072)