Interactive Layout-Togglable Storytelling

• Layout-togglable storytelling is a visual narrative approach that allows users to interactively edit spatial compositions using explicit, editable layout representations.
• It leverages techniques such as bounding boxes, panel templates, and aspect-ratio-aware masks to reframe scenes without reprocessing the entire image pipeline.
• These systems balance deterministic recomposition and conditional regeneration to preserve text alignment, character consistency, and style in dynamic narrative editing.

Searching arXiv for the cited storytelling/layout-control papers to ground the article. arxiv_search.query({"search_query":"id:(Wang et al., 2023) OR id:(Gong et al., 2023) OR id:(Wang et al., 2023) OR id:(Garcia-Dorado et al., 2017)","max_results":10,"sort_by":"submittedDate","sort_order":"descending"}) to=arxiv_search.query 天天彩票软件ూ ,一本道 天天中彩票未json {"result":[{"arxiv_id":"(Wang et al., 2023)","version":"v1","idv":"(Wang et al., 2023)v1","title":"MagicScroll: Nontypical Aspect-Ratio Image Generation for Visual Storytelling via Multi-Layered Semantic-Aware Denoising","categories":"cs.CV cs.GR","published":"2023-12-18","updated":"2023-12-18","pdf_url":"http://arxiv.org/pdf/([2312.10899](/papers/2312.10899))v1","abs_url":"https://arxiv.org/abs/([2312.10899](/papers/2312.10899))v1"},{"arxiv_id":"([2311.11243](/papers/2311.11243))","version":"v1","idv":"([2311.11243](/papers/2311.11243))v1","title":"AutoStory: Generating Diverse Storytelling Images with Minimal Human Effort","categories":"cs.CV","published":"2023-11-19","updated":"2023-11-19","pdf_url":"http://arxiv.org/pdf/([2311.11243](/papers/2311.11243))v1","abs_url":"https://arxiv.org/abs/([2311.11243](/papers/2311.11243))v1"},{"arxiv_id":"([1712.06654](/papers/1712.06654))","version":"v2","idv":"([1712.06654](/papers/1712.06654))v2","title":"Graphic Narrative with Interactive Stylization Design","categories":"cs.GR cs.CV","published":"2017-12-18","updated":"2018-06-25","pdf_url":"http://arxiv.org/pdf/([1712.06654](/papers/1712.06654))v2","abs_url":"https://arxiv.org/abs/([1712.06654](/papers/1712.06654))v2"},{"arxiv_id":"([2305.18247](/papers/2305.18247))","version":"v2","idv":"([2305.18247](/papers/2305.18247))v2","title":"TaleCrafter: Interactive Story Visualization with Multiple Characters","categories":"cs.CV","published":"2023-05-29","updated":"2023-11-29","pdf_url":"http://arxiv.org/pdf/([2305.18247](/papers/2305.18247))v2","abs_url":"https://arxiv.org/abs/([2305.18247](/papers/2305.18247))v2"}]} Layout-togglable storytelling denotes visual storytelling systems in which spatial composition remains editable after the story content, selected frames, or style specification have been established. In the cited literature, this capability is implemented through editable object bounding boxes, panel-template arrays, or aspect-ratio-aware layout masks, so that a story can be re-framed, re-paneled, or re-generated without reconstructing the entire pipeline from scratch. Across recent systems, the technical objective is consistent: preserve text alignment, maintain character or scene consistency, and support rapid interaction at inference time while treating layout as an explicit control signal rather than an implicit by-product of prompting (Wang et al., 2023, Gong et al., 2023, Wang et al., 2023, Garcia-Dorado et al., 2017).

1. Conceptual scope and research lineage

An early formulation of layout toggling appears in storyboard generation from existing image sets. "Graphic Narrative with Interactive Stylization Design" converts a video clip or photo album into a storyboard by selecting content images, placing them into panels, and applying stylization, then lets the user instantly re-layout the same selected frames into different storyboard templates at interactive rates (Garcia-Dorado et al., 2017). In that setting, layout toggling is primarily a re-composition problem: the system ships with 70+ hand-designed layouts, each template encodes panel positions, proportions, gutters, and reading order, and the same stylized bitmaps are simply redrawn into a new array of panel rectangles.

In diffusion-era story visualization, layout toggling becomes a conditional generation problem. "TaleCrafter" separates story-to-prompt generation, text-to-layout generation, controllable text-to-image generation, and image-to-video animation, explicitly emphasizing interactive editing of bounding-box layouts (Gong et al., 2023). "AutoStory" extends this idea into an automated story visualization system that uses LLMs for layout planning and text-to-image models for image synthesis, with bounding boxes exposed in a GUI for drag-and-resize editing at inference time (Wang et al., 2023). "MagicScroll" generalizes layout control to nontypical aspect-ratio image generation, where GPT-produced boxes are rasterized into masks and then geometrically remapped across scrolls, panoramas, and comic-strip formats (Wang et al., 2023).

Taken together, these systems define layout toggling as a family of interaction patterns rather than a single algorithm. In one branch, the user reassigns already-rendered images to new panels. In another, the user edits sparse scene structure and the model re-synthesizes the affected frame. A plausible implication is that the term covers both deterministic re-composition and conditional re-generation, provided that the layout representation is explicit, editable, and causally linked to the output image.

2. Layout representations and control abstractions

The central technical choice is the representation of layout. In AutoStory, each panel layout is

$$\sigma_i \;=\;\Bigl\{\,p_i^{\rm global},\;(p_{i1}^{\rm local},b_{i1}),\dots,(p_{i k_i}^{\rm local},b_{i k_i})\Bigr\},$$

where $p_i^{\rm global}$ is a panel-level prompt and each $(p_{ij}^{\rm local}, b_{ij})$ couples a local text phrase with a bounding box. These sparse bounding boxes form the layout control “canvas,” and they are deliberately simple to drag or resize (Wang et al., 2023).

TaleCrafter formalizes layout as a set of quantized object boxes with category labels,

$$\mathcal{L}=\bigl\{(x_i,y_i,w_i,h_i,l_i)\bigr\}_{i=1}^N,$$

where $(x,y,w,h)$ are quantized and $l_i$ is an Object365 category. This layout is predicted by a discrete-diffusion text-to-layout model derived from LayoutDM, and the resulting representation is suitable for partial conditioning during interactive editing (Gong et al., 2023).

MagicScroll moves from boxes to masks. GPT-3.5 produces bounding boxes for scenes and objects, $\{b^{(i)}_{\text{scn}}, b^{(j)}_{\text{obj}}\}$, which are rasterized into binary masks $M_{\text{scn}}, M_{\text{obj}}$. These masks are then injected as explicit conditioning streams during denoising, alongside text and reference-image signals (Wang et al., 2023). Graphic Narrative uses a different abstraction entirely: layout is a predefined template consisting of panel rectangles, and toggling amounts to swapping one template for another (Garcia-Dorado et al., 2017).

System	Editable layout primitive	Toggle operation
AutoStory	$(p_{ij}^{\rm local}, b_{ij})$ per panel	drag/resize boxes; rescale dense guides; regenerate panel
TaleCrafter	$(x_i,y_i,w_i,h_i,l_i)$	fix subset; resample remaining boxes
MagicScroll	$p_i^{\rm global}$0	scale or interpolate masks across formats
Graphic Narrative	template panel rectangles	redraw pre-stylized images in new template

This diversity of representations clarifies a common misconception: layout toggling is not identical to object-level box editing. In Graphic Narrative, the editable variable is the storyboard template; in MagicScroll, it is the aspect-ratio-aware scene mask; in AutoStory and TaleCrafter, it is object placement itself.

3. End-to-end system architectures

AutoStory implements a three-stage pipeline. First, a user types a story prompt or a single-sentence description $p_i^{\rm global}$1. If $p_i^{\rm global}$2 is not a full story, an LLM expands it into full story text $p_i^{\rm global}$3. The LLM then splits $p_i^{\rm global}$4 into $p_i^{\rm global}$5 panels and produces a layout specification for each panel,

$p_i^{\rm global}$6

Second, for each local prompt, the system generates a subject image $p_i^{\rm global}$7, detects and masks the object via Grounding-DINO→SAM, extracts either a sketch via PidiNet or keypoints via HRNet, and stamps the resulting dense condition $p_i^{\rm global}$8 back into the corresponding box $p_i^{\rm global}$9 to form a panel-level dense map $(p_{ij}^{\rm local}, b_{ij})$0. Third, the diffusion model is re-run under both sparse box control and dense control, with character-specific LoRA modules $(p_{ij}^{\rm local}, b_{ij})$1, region-sampled cross-attention, and a lightweight T2I-Adapter (Wang et al., 2023).

TaleCrafter decomposes the problem into four sequential modules: story-to-prompt (S2P), text-to-layout (T2L), controllable text-to-image (C-T2I), and image-to-video (I2V). S2P uses GPT-4 prompt engineering to rewrite or expand a story into $(p_{ij}^{\rm local}, b_{ij})$2 concise image prompts. T2L uses a discrete-diffusion model based on LayoutDM to predict object layouts from those prompts. C-T2I modifies Stable Diffusion v1.4 so that its U-Net accepts three condition streams in parallel—text, layout, and sketch—together with LoRA-based identity adapters. I2V then estimates a depth map via 3D-photography, warps to novel camera paths, and optionally overlays TTS audio (Gong et al., 2023).

MagicScroll places layout prediction inside a broader framework for arbitrary aspect-ratio generation. Its architecture consists of GPT-based layout prediction, multi-layered semantic-aware denoising with latent smoothing and blending, and style-control modules. At each denoising step, the U-Net receives the current latent $(p_{ij}^{\rm local}, b_{ij})$3, semantic-layer latents $(p_{ij}^{\rm local}, b_{ij})$4, and cross-attention keys and values derived from text embeddings, reference-image latents, and layout masks (Wang et al., 2023).

Graphic Narrative, by contrast, is not a diffusion system. Its pipeline samples candidate frames from video or photo collections, removes near-duplicates using a 64-bit perceptual hash, filters blurry images using the sharpness score $(p_{ij}^{\rm local}, b_{ij})$5, orders frames by timestamp or user-driven importance, assigns them to panels in a selected template, and crops each image so that the window aspect ratio best matches the target panel aspect ratio $(p_{ij}^{\rm local}, b_{ij})$6 by minimizing $(p_{ij}^{\rm local}, b_{ij})$7. Stylization is then applied with interactive or procedural filter sequences implemented in Halide (Garcia-Dorado et al., 2017).

4. Inference-time toggling mechanisms

AutoStory’s layout-toggling mechanism is explicit and panel-local. Once the LLM produces the set of boxes $(p_{ij}^{\rm local}, b_{ij})$8, they are rendered in a simple GUI. At generation time, the user can select any box and drag or resize it. Internally, the system updates the sparse specification $(p_{ij}^{\rm local}, b_{ij})$9 by replacing $$\mathcal{L}=\bigl\{(x_i,y_i,w_i,h_i,l_i)\bigr\}_{i=1}^N,$$0 with $$\mathcal{L}=\bigl\{(x_i,y_i,w_i,h_i,l_i)\bigr\}_{i=1}^N,$$1, automatically re-rescales and translates each dense patch $$\mathcal{L}=\bigl\{(x_i,y_i,w_i,h_i,l_i)\bigr\}_{i=1}^N,$$2 from its old box to the new box, composites the updated dense map $$\mathcal{L}=\bigl\{(x_i,y_i,w_i,h_i,l_i)\bigr\}_{i=1}^N,$$3, and re-invokes the diffusion pass on panel $$\mathcal{L}=\bigl\{(x_i,y_i,w_i,h_i,l_i)\bigr\}_{i=1}^N,$$4 only. This is entirely feed-forward, with no LLM or fine-tuning calls, and the new boxes immediately constrain both region-sampled cross-attention and the dense adapter, so users see the refreshed story frame in a few seconds (Wang et al., 2023).

The corresponding attention constraint is localized. For coordinates $$\mathcal{L}=\bigl\{(x_i,y_i,w_i,h_i,l_i)\bigr\}_{i=1}^N,$$5, AutoStory applies

$$\mathcal{L}=\bigl\{(x_i,y_i,w_i,h_i,l_i)\bigr\}_{i=1}^N,$$6

so latents inside each bounding box attend only to the corresponding local text embedding. The dense condition is injected into each U-Net block through

$$\mathcal{L}=\bigl\{(x_i,y_i,w_i,h_i,l_i)\bigr\}_{i=1}^N,$$7

with $$\mathcal{L}=\bigl\{(x_i,y_i,w_i,h_i,l_i)\bigr\}_{i=1}^N,$$8 denoting the T2I-Adapter (Wang et al., 2023).

TaleCrafter implements toggling through partial conditioning in discrete diffusion. The user may drag, resize, delete, or add boxes in a simple GUI. The retained entries are indicated by a mask $$\mathcal{L}=\bigl\{(x_i,y_i,w_i,h_i,l_i)\bigr\}_{i=1}^N,$$9, and at each reverse step the current noisy layout state is mixed with the user-fixed coordinates: $(x,y,w,h)$0 A reverse-diffusion step then resamples only the unconstrained components. The entire re-sampling takes only a few seconds on a single GPU (Gong et al., 2023). This means layout toggling here is not merely geometric editing; it is a constrained re-sampling process that can preserve chosen placements while regenerating alternatives for the rest.

MagicScroll’s toggle mechanism is geometric rather than box-by-box semantic re-sampling. It decouples canonical scroll layouts from target formats using scaling: $\{b^{(i)}_{\text{scn}}, b^{(j)}_{\text{obj}}\}$3 and supports smooth interpolation between aspect ratios through

$(x,y,w,h)$1

Comic-strip paneling is implemented by subdividing the scroll into a grid of equal-sized panels and reassigning object boxes to the nearest panel anchor (Wang et al., 2023).

Graphic Narrative represents the opposite endpoint: the same set of selected frames and styling is re-laid-out in a different template by re-composing pre-stylized bitmaps into a new array of panel rectangles. Because layout is a simple array of rectangles, layout swap cost is $(x,y,w,h)$2 ms, and end-to-end “shuffle storyboard” latency on device is approximately $(x,y,w,h)$3–$(x,y,w,h)$4 ms, yielding approximately $(x,y,w,h)$5 fps update rate for users (Garcia-Dorado et al., 2017).

5. Consistency, identity, and style under layout edits

Layout toggling alone does not guarantee story coherence. The modern systems therefore add separate mechanisms for identity consistency, local structure, and style preservation.

AutoStory uses a simple yet effective method to generate multi-view consistent character images. Starting from one exemplar $(x,y,w,h)$6, it applies a 3D-aware image translator $(x,y,w,h)$7, such as One-2-3-45 or Zero-1-to-3, to produce viewpoint variations

$(x,y,w,h)$8

These images are stacked as if they were frames of a video, and standard self-attention

$(x,y,w,h)$9

is replaced by

$l_i$0

where $l_i$1 is the first frame and $l_i$2 is the previous one, so all viewpoints share identity features. The resulting sketches or keypoints are then used to fine-tune a small character LoRA $l_i$3, and all characters’ LoRAs are merged with gradient fusion during story synthesis (Wang et al., 2023).

TaleCrafter preserves identity through a learned token $l_i$4 and low-rank adapters in self-attention and cross-attention layers. For a frozen weight matrix $l_i$5, the adapted weight is

$l_i$6

At inference, category words such as “dog” are replaced by “<dog\_id> dog” in prompts and box labels. Multi-character composition is handled by iterative inpainting: generate the background plus one character, mask the next character’s box, and inpaint that region using the corresponding adapters (Gong et al., 2023).

MagicScroll organizes consistency through semantic layers. It splits the latent representation into foreground objects, midground scenes, and background style, and uses latent blending to schedule which layer dominates over time. Empirically, steps $l_i$7–$l_i$8 are object-heavy, while steps $l_i$9–$\{b^{(i)}_{\text{scn}}, b^{(j)}_{\text{obj}}\}$0 are scene-heavy with periodic background injections. Latent smoothing then solves an overlapping sliding-window regression for spatial coherence,

$\{b^{(i)}_{\text{scn}}, b^{(j)}_{\text{obj}}\}$1

with edge masking via $\{b^{(i)}_{\text{scn}}, b^{(j)}_{\text{obj}}\}$2 (Wang et al., 2023).

Graphic Narrative achieves style stability by decoupling styling from layout. All stylization is done once per frame; layout toggling only re-draws the pre-stylized bitmaps into a new template. This suggests that in non-generative pipelines, layout variation can be made almost orthogonal to style variation, whereas in diffusion pipelines additional conditioning modules are needed to prevent layout edits from perturbing identity or appearance (Garcia-Dorado et al., 2017).

6. Empirical findings, evaluation practices, and interpretive cautions

AutoStory reports an ablation on control signals. With no control, objects go missing and attributes blend. With boxes only, attribute blending is fixed but small objects are still omitted. With dense only, all objects are picked up but attribute confusion returns. With boxes plus dense, omissions and confusion are eliminated (Wang et al., 2023). In a user study with 32 participants on a Likert 1–5 scale, Text Alignment improved from 2.19 and 2.17 for Custom-Diffusion and Paint-by-Example to 4.31, Consistency from 2.64 and 2.53 to 4.16, and Quality from 2.65 and 2.35 to 4.08. Quantitative CLIP scores averaged over 71 panels report Text–Image Sim of 0.7721 versus 0.7332 and 0.7172, and Image–Image Sim of 0.6748 versus 0.6402 and 0.6214 (Wang et al., 2023).

TaleCrafter evaluates 700 images over 35 prompts using CLIP-based text–image alignment and identity consistency. The reported results are 0.7676 and 0.6758 for the proposed method, compared with 0.7422 and 0.6323 for Custom-Diffusion and 0.7087 and 0.6104 for Paint-by-Example. In a human preference study with 50 users over 9 stories × 3 prompts, average scores on a 1–3 scale are 2.873 for text–image correspondence, 2.651 for character coherence, and 2.725 for quality, versus lower baselines for the compared methods (Gong et al., 2023).

MagicScroll contributes a benchmark for nontypical aspect-ratio image generation for visual storytelling, totalling approximately 6,000 samples across Painting, Comic Strips, and Cinematic Panoramas. Its custom metrics include Layout Consistency Score (mean IoU across predicted versus GPT-reference masks), Narrative Alignment Score, Local-Global Image Score, Global Embedding Variance, and Edge Aesthetic. On the Painting set, MagicScroll reports CSGT of 0.80 versus 0.78 for StableDiffusion and 0.76 for MultiDiffusion; LGIS of 0.53 versus 0.60 and 0.57; GEV of 0.057 versus 0.049 and 0.051; LCS of 0.84 versus 0.72 for GLIGEN; and NAS of 0.67 versus 0.58 for ControlNet (Wang et al., 2023).

Graphic Narrative emphasizes runtime rather than text-image alignment metrics. On a 1920×1080 image, stylization on desktop takes approximately 70–100 ms per style and 180–300 ms per style on a Nexus 6P; SSD face/object detection runs in approximately 10 ms on device; and layout swap cost is less than 5 ms (Garcia-Dorado et al., 2017). This highlights an important distinction in evaluation culture: classical storyboard systems prioritize interactive rates and mobile deployment, whereas diffusion-based systems emphasize semantic alignment, identity preservation, and user preference.

A common misconception is that layout toggling necessarily requires global optimization or re-training. The record in these papers is more heterogeneous. Graphic Narrative explicitly uses no global layout optimizer or ranking, relying instead on pre-ranked designer templates (Garcia-Dorado et al., 2017). AutoStory states that layout edits are handled by a single forward pass with no re-training or slow optimization loops (Wang et al., 2023). TaleCrafter re-samples only the unconstrained subset of boxes under partial conditioning (Gong et al., 2023). MagicScroll uses simple geometric transforms and interpolation for aspect-ratio toggling (Wang et al., 2023). This suggests that the computational burden of toggling depends less on the existence of layout control than on where the layout variable sits in the pipeline: template composition, constrained layout diffusion, or full image diffusion.

Another misconception is that sparse control and dense control are interchangeable. AutoStory explicitly distinguishes them: sparse control conditions such as bounding boxes are suitable for layout planning, while dense control conditions such as sketches and keypoints are suitable for generating high-quality image content (Wang et al., 2023). The broader implication is that layout-togglable storytelling often depends on a hierarchical control stack rather than a single representation, with coarse geometry used for interaction and denser structure used for synthesis fidelity.