Semantic Prompting: Agentic Incremental Narrative Refinement through Spatial Semantic Interaction

Abstract: Interactive spatial layouts empower users to synthesize information and organize findings for sensemaking. While LLMs can automate narrative generation from spatial layouts, current collage-based and re-generation methods struggle to support the incremental spatial refinements inherent to the sensemaking process. We identify three critical gaps in existing spatial-textual generation: interaction-revision misalignment, human-LLM intent misalignment, and lack of granular customization. To address these, we introduce Semantic Prompting, a framework for spatial refinement that perceives semantic interactions, reasons about refinement intent, and performs targeted positional revisions. We implemented S-PRISM to realize this framework. The empirical evaluation demonstrated that S-PRISM effectively enhanced the precision of interaction-revision refinement. A user study ($N=14$) highlighted how participants leveraged S-PRISM for incremental formalization through interactive steering. Results showed that users valued its efficient, adaptable, and trustworthy support, which effectively strengthens human-LLM intent alignment.

• The paper presents Semantic Prompting and S-PRISM as a novel framework for agentic narrative refinement that addresses inefficiencies in spatial-to-text conversion.
• It employs a hierarchical agent pipeline to translate spatial user actions into targeted narrative updates, enhancing precision and semantic fidelity.
• Empirical evaluations and user studies demonstrate improved intent alignment, incremental correctness, and greater customization over traditional methods.

Semantic Prompting: An Agentic Framework for Incremental Narrative Refinement via Spatial Semantic Interaction

Introduction and Motivation

The paper presents Semantic Prompting, a novel framework and accompanying system (S-PRISM) for agentic, incremental narrative refinement driven by semantic spatial interactions. This approach directly addresses critical inefficiencies and usability gaps in existing LLM-powered spatial-to-text systems—most notably, interaction-revision misalignment, human-LLM intent misalignment, and inflexible granular customization. The research re-envisions the sensemaking workflow by leveraging spatial layouts as externalized cognition spaces, where human analytical actions are systematically translated into precise narrative refinements via agentic reasoning.

Traditional methods for connecting spatial scenes and text—such as full regeneration, collage-based refinement, and prompt engineering—either induce excessive rewording, demand explicit prompt-writing skill, or lack support for positionally granular, intent-aligned editing. By introducing explicit agent-based reasoning over semantic spatial interactions, Semantic Prompting offers fine control: spatial manipulations are interpreted as signals of intent and mapped onto deterministic updates, enhancing both transparency and user agency.

Related Work and Gaps in Existing Systems

Current approaches to spatial-to-narrative conversion include prompt engineering [obeid2020chart], collage-based transformations [buschek2024collage], and regeneration-centric pipelines [tang2025respire]. These techniques suffer from several limitations:

• Interaction-Revision Misalignment: Minor spatial edits yield global narrative rewrites, destabilizing user mental models.
• Human-LLM Intent Misalignment: Opaque “black box” LLM outputs impede user verification and trust, especially given a lack of transparency into reasoning chains.
• Inflexible Customization: Workspace semantics are not fully or flexibly leveraged, constraining targeted, granular steering of output narratives.

Semantic Interaction (SI) [endert2012semantic] has shown that spatial user actions can serve as high-level intent cues for model steering, but existing SI frameworks suffer from shallow intent modeling or rely on expensive, task-specific heuristics. The advent of LLMs’ robust reasoning (e.g., chain-of-thought [wei2022chain]) motivates a fundamental shift towards real-time, granular, and user-transparent refinement linked explicitly to spatial actions.

Figure 1: Previous frameworks focus on manual prompt engineering or spatial-to-report generation; Semantic Prompting introduces agentic translation of semantic spatial interactions into incremental narrative revisions.

The Semantic Prompting Framework and S-PRISM System

The Semantic Prompting framework operationalizes the concept of using rich spatial semantics as active drivers for narrative change, implemented in the S-PRISM system. The core architecture features a hierarchical agent pipeline:

• Intent Inferencer: Analyzes deltas in spatial configuration (e.g., highlights, notes, clusters) and deduces user narrative intent.
• Refining Agent: Executes localized modifications, restricted to sections that align with inferred user targets, minimizing unintended rewording and maintaining structural consistency.

The agentic workflow is quadripartite: Interact (spatial manipulations by the user), Perceive (system captures deltas), Reason (agents infer intent), and Act (perform precise narrative updates). Triggering of refinement remains user-initiated, preserving agency and enabling batch or micro-edit cycles.

Figure 2: S-PRISM’s multi-agent pipeline, mapping user-driven spatial manipulations to agent-inferred narrative updates with transparent intent surfacing and precise result highlighting.

S-PRISM’s interface supports direct manipulation through frames (for structure), highlights (for emphasis/extraction), and notes (for meta-prompts and corrections), with supporting configuration panels for template, LLM model, and visible diff-tracking of revisions.

Figure 3: The S-PRISM interface illustrates spatial organization on the left and stepwise, transparent refinement logic and control settings on the right.

Empirical Evaluation

A technical evaluation was conducted using the “Sign of the Crescent” sensemaking benchmark, leveraging GPT-4o-mini for consistency. S-PRISM was compared with a regeneration-from-scratch baseline (ReSPIRE [tang2025respire]) on annotated test cases targeting two axes:

• Targeted Refinement: Precision and recall for localized paragraph edits corresponding only to user-specified spatial changes.
• Semantic Fidelity: Alignment between revision content and explicit semantic markers (entities, key terms) from interaction deltas.

S-PRISM exhibited a marked improvement in precision for both targeted and semantic fidelity metrics (0.951 and 0.558, respectively), yielding stronger F1 scores than the regeneration baseline.

User Study

A 14-participant study was performed (diverse in spatial interface expertise, all advanced LLM users) over sequential and open-ended tasks, examining steering efficacy, intent alignment, and customization. Key findings include:

• Incremental Formalism: S-PRISM enabled a monotonic increase in correctness across iterative refinements, outperforming regeneration methods, which plateaued or regressed in complex tasks.
• Intent Transparency and Alignment: Surfacing agent reasoning was universally valued; participants successfully used new spatial actions (notably, notes for meta-instructions) to rectify or specify narrative goals. Highlight-only strategies were sometimes insufficient for fine placement, but escalations via notes provided effective explicit control.
• Granular Customization: Participants utilized spatial nesting and structuring (frames, subframes) to reshape narrative hierarchy and employed notes to control both macro and micro narrative aspects. This facilitated cross-task transitions (e.g., analytical report to itinerary) even without explicit prompt editing.

  Figure 4: Phase I sensemaking tasks illustrate user-driven mapping from spatial organization to incrementally refined, evidence-grounded reports.

  

  Figure 5: Interaction logs show steady increase in report correctness per task, with action diversity broadening in more advanced use cases.

  

  Figure 6: Subjective ratings confirm strong trust in reasoning transparency, refinement effectiveness, and system confidence.

  

  Figure 7: Example user workspace evolution demonstrates dynamic restructuring and detailed customization enabled by Semantic Prompting.

Discussion and Implications

Semantic Prompting shifts LLM-assisted writing beyond template-based and collage paradigms, leveraging direct mapping from spatial schemas and meta-instructions to narrative outcome. The agentic, transparent pipeline facilitates intent alignment, preserves interaction-revision stability, and supports iterative, user-driven workflows central to complex sensemaking. The framework generalizes to a range of knowledge synthesis applications—policy, travel, intelligence analysis—where users’ analytic priorities are diverse and evolve dynamically.

However, high-density interaction scenarios can surface “attention dilution” in LLM behavior, and mapping remains imperfect in situations where user logic diverges significantly from agent inference, evidencing a need for deeper semantic modeling and mixed-initiative support.

Conclusion

Semantic Prompting and S-PRISM demonstrate that agentic semantic interaction enables precise, intent-transparent, and user-steerable narrative refinement in spatial sensemaking environments. This system achieves high empirical precision in mapping user actions to text, resolves core usability gaps in LLM-human collaboration, and supports both structural and fine-grained customization. The approach suggests future research directions in action-to-goal alignment, adaptive agent transparency, and further scaling of interactive, multi-agent architectures for broader domains and workflows.