SMART-Editor: State-Space Editing Framework

• SMART-Editor is a multi-agent framework defined by its explicit state space that preserves semantic coherence and global structural integrity across edits.
• It decomposes editing into agents (Action, Critique, Optimizer) that translate natural-language instructions into symbolic edit programs, ensuring controlled, reward-optimized transformations.
• The framework is applied in diverse domains including poster design, smart-contract development, latent 3D/2D image editing, accessible equation authoring, and collaborative rich-text editing.

SMART-Editor denotes, most explicitly, a multi-agent framework for compositional layout and content editing across structured domains such as posters and websites and unstructured domains such as natural images. Across adjacent work, the same label also characterizes a broader design pattern: editing systems that interpose a structured intermediate representation, learned reasoning layer, or both between an instruction and the final artifact, so that edits preserve semantic coherence, structural integrity, or security properties rather than applying only local transformations. In the supplied literature, this pattern appears in reward-guided layout editing, finite-state smart-contract design, latent 3D editing, accessible equation authoring, mask-free image editing, and multiplexed collaborative rich-text editing (Mondal et al., 30 Jul 2025, Mavridou et al., 2018, Chen et al., 2023, Ge et al., 2 Jan 2025, Sun et al., 17 Apr 2025, Jatana et al., 2024).

1. Scope and defining characteristics

A SMART-Editor system is defined less by modality than by the representation on which it edits. In structured design editing, the operative state is a layout-content pair $(L, C)$ plus a symbolic Action Registry. In secure contract editing, it is an FSM with states, guards, actions, transitions, and tags. In latent 3D editing, it is a Shap-E latent code. In accessible mathematics, it is an AST coupled to a grid model. In mask-free image editing, it is a set of region-aware tokens, reasoning masks, and hypergraph-enhanced features. In collaborative text editing, it is a collection of Y.Doc-backed doclets routed over a single multiplexed socket (Mondal et al., 30 Jul 2025, Mavridou et al., 2018, Chen et al., 2023, Ge et al., 2 Jan 2025, Sun et al., 17 Apr 2025, Jatana et al., 2024).

System	Domain	Primary editing substrate
SMART-Editor	Posters, websites, natural images	Layout-content state and action plans
FSolidM	Ethereum smart contracts	Finite state machines
Shap-Editor	Instruction-guided 3D editing	Shap-E latent space
StereoMath	Accessible equation editing	AST plus grid model
SmartFreeEdit	Instruction-driven image editing	Region-aware tokens and reasoning masks
Differentially processed collaborative editor	Rich-text collaboration	Y.Doc doclets over one WebSocket

This suggests that SMART-Editor is best understood as an architectural family rather than a single UI style. The common denominator is the use of an explicit state space that can be validated, optimized, or constrained before rendering, code generation, or synchronization.

2. Reward-guided compositional editing in structured and unstructured media

The most direct instantiation is the framework titled "SMART-Editor: A Multi-Agent Framework for Human-Like Design Editing with Structural Integrity" (Mondal et al., 30 Jul 2025). It decomposes editing into an Action Agent, a Critique Agent, and an Optimizer Agent. The Action Agent receives a raw image or layout, a parsed layout $L$ with objects $o_i$ and bounding boxes $b_i = (x_i, y_i, x_i', y_i')$, associated content $C$, and an instruction $E$, then converts an LLM-generated natural-language plan into a symbolic edit program over operations such as translate, insert, remove, reorder, resize, align, snap-to-grid, and update_object_text. The Critique Agent scores the edited result along interpretable axes. For structured domains these include Edit Adherence, Narrative Coherence, Cross-Sectional Consistency, Overlap penalty, Whitespace penalty, and Alignment reward; for natural images they include Edit Adherence, Semantic Match, Object Size and Overlap Realism, and Depth Layering and Occlusion. The Optimizer Agent either runs Reward-Refine, an inference-time iterative refinement loop, or RewardDPO, a training-time preference optimization procedure built from reward-aligned layout pairs.

Its reward formalization is explicitly compositional:

$$R(L', C') = \sum_{k=1}^{K} \lambda_k \cdot r_k(L', C').$$

For structured layouts, the semantic and spatial subterms include

$$r_{\mathrm{sem}} = 1 - \frac{v}{|O|}, \qquad r_{\mathrm{cross}} = 1 - \frac{c}{|G|}, \qquad r_{\mathrm{align}} = \frac{1}{N}\sum_{i=1}^{N}\mathbb{I}[x_i \bmod \mathrm{grid} = 0],$$

together with a whitespace term $1 - A_{\mathrm{used}}/A_{\mathrm{canvas}}$ and overlap penalties. Reward-Refine performs beam-searched proposal-selection over evolving states $(L^{(t)}, C^{(t)})$, while RewardDPO optimizes a one-shot policy with

$L$0

Evaluation is centered on SMARTEdit-Bench, which contains 1,200 synthetic poster edits and 250 expert-tagged gold posters, 564 real-world website triplets from GitHub commits plus 510 synthetic website triplets, and 500 HQ-Edit natural-image examples. RewardDPO achieves up to 15% gains in structured settings, while Reward-Refine is advantageous on natural images. On posters, GPT-4o + Reward-Refine reaches EA 4.70, Narrative 0.47, Cross-sectional 0.67, Overlap 52, Whitespace 3.2, and Alignment 36.5. On websites, GPT-4o + Reward-Refine reaches SC 4.50 and LS 4.45. On natural images, GLIGEN + RewardDPO reaches EA 4.35, DL 4.20, SM 4.40, and Overlap 4.25, outperforming DALLE-3, HIVE, and InstructPix2Pix on the reported axes. Human evaluation reports RewardDPO vs Zero-Shot win rates of 95.5% on posters and 93.7% on websites, while Reward-Refine vs Zero-Shot reaches 92.1% on natural images (Mondal et al., 30 Jul 2025).

3. Formal semantics and secure-by-design editing

FSolidM shows a markedly different but structurally analogous interpretation of SMART-Editor: model-driven editing for Ethereum smart contracts (Mavridou et al., 2018). It is an open-source, web-based framework built on WebGME that allows developers to define contracts as finite state machines with rigorous and clear semantics, then generate Solidity automatically. The editor enforces exactly one InitialState, supports multiple States, and represents each transition by a name, guards, input and output data, statements, and tags. Timed transitions are separate elements with guards, statements, and a time in seconds since contract creation at which the transition is automatically executed if the guard holds.

Its compact formalization is

$L$1

with timed extensions

$L$2

Guards are predicates $L$3 over contract and input variables, and actions are sequences over contract state and outputs. Code generation maps states to a Solidity enum and a state variable, transitions to Solidity functions, guards to conjuncted require(...) statements, actions to emitted statements, and state changes to assignments. The precondition–postcondition scheme is stated as

$L$4

followed by

$L$5

FSolidM’s built-in plugins operationalize security patterns. The Locking plugin injects a locked field and a modifier that prevents reentrancy. The Transition Counter plugin adds a nextTransitionNumber parameter and enforces a strictly increasing counter to mitigate transaction ordering dependence. Automatic Timed Transitions execute all due timed transitions before any invoked non-timed transition. Access Control injects administrator checks into tagged transitions. The framework also synchronizes model and code views, locks generated code regions, syntax-checks developer fragments with the ConsenSys solidity-parser, validates FSM correctness, and supports collaborative versioning with branching, merging, and history views.

The blind auction case study uses four states—AcceptingBlindedBids, RevealingBids, Finished, and Canceled—and transitions such as bid, close, reveal, finish, cancelABB, cancelRB, unbid, and withdraw. Gas measurements compiled with Solidity 0.4.17 and optimizer enabled quantify the overhead of security plugins. Deployment costs are 504,672 gas with no plugins, 577,514 with Locking, 562,800 with Transition Counter, and 637,518 with both. For selected transitions, overhead is nearly constant per plugin and additive: for bid, 58,249 / 68,917 / 63,924 / 74,607 gas across none / locking / counter / both; for withdraw, 20,290 / 30,961 / 25,961 / 36,629. The framework reduces the semantic gap between contract intent and contract code, but it does not claim machine-checked proofs for generated code, and broader properties are deferred to future integration with model checkers such as D-Finder and nuXmv (Mavridou et al., 2018).

4. Latent and diffusion-based visual SMART-Editors

Shap-Editor and SmartFreeEdit extend the SMART-Editor pattern into 3D and 2D generative editing, respectively, but they do so with different intermediate states and optimization strategies (Chen et al., 2023, Sun et al., 17 Apr 2025). Shap-Editor is a feed-forward 3D editor that operates in the latent space of Shap-E. A 3D asset $L$6 is encoded into a latent tensor $L$7, and editing is expressed as

$L$8

where $L$9 edits the latent conditioned on an instruction $o_i$0. The editor is initialized from the Shap-E text-conditional diffusion denoiser and receives a concatenation of a noisy branch and the clean source latent,

$o_i$1

with $o_i$2 fixed. Training distills text-guided image-to-image and text-to-image diffusion priors via SDS, together with depth-preserving and masked photo/depth regularizers. The result is a single universal latent editor function that performs an edit in approximately one second per forward pass, excluding rendering and any 3D encoding or decoding overhead. On a 20-pair evaluation, local editing reaches CLIP_sim 0.292 and CLIP_dir 0.097 in the single-prompt setting, outperforming Vox-E, IN2N, and Text2Mesh on the reported local metrics; global editing reaches CLIP_sim 0.272, CLIP_dir 0.070, and Structure Dist. 0.008, with edit times around one second versus roughly 14–40 minutes for the test-time-optimization baselines.

SmartFreeEdit addresses instruction-driven 2D image editing without user-provided masks. Its pipeline is natural-language instruction $o_i$3 MLLM reasoning $o_i$4 reasoning segmentation $o_i$5 hypergraph-augmented inpainting (Sun et al., 17 Apr 2025). Spatial grounding is carried by region-aware tokens,

$o_i$6

combining region visual features, positional encodings, and mask embeddings. Mask generation is driven by a <seg> token whose hidden state is passed through an MLP and a SAM-like pixel decoder. The system conceptually scores candidate regions with a combination of IoU to the decoder mask, CLIP-based region-text alignment, and spatial-relation consistency. Its inpainting module inserts a hypergraph operator into the VAE encoder and decoder mid-blocks, with propagation of the form

$o_i$7

so that high-order relations among distant regions influence the final synthesis.

On Reason-Edit, SmartFreeEdit reports, for understanding scenarios, PSNR 28.99, SSIM 0.92, LPIPS $o_i$8, CLIPSim 24.43, and Ins-Align 0.86; for reasoning scenarios, PSNR 31.27, SSIM 0.92, LPIPS $o_i$9, and CLIPSim 21.46. On BrushBench, the blended model reports IR 13.39, HPS 28.50, and AS 6.48 while maintaining strong LPIPS, MSE, and CLIPSim. SmartFreeEdit therefore differs from Shap-Editor in two important respects: it remains image-native rather than latent-3D-native, and it relies on reasoning segmentation and hypergraph context propagation rather than amortized 3D latent editing.

5. Accessibility, interaction design, and collaborative state management

StereoMath shows that SMART-Editor need not be primarily visual (Ge et al., 2 Jan 2025). It is an accessible equation editor for blind and low-vision users that treats equation editing as a spatial, musical interaction. Its document model combines an AST with a grid representation: internal nodes correspond to constructs such as frac, sqrt, superscript, subscript, sum, integral, and matrix, while the same structure is also rendered as a 2D table for navigation and audio. Spatial audio maps horizontal location to stereo pan,

$b_i = (x_i, y_i, x_i', y_i')$0

and vertical position plus nesting depth to pitch, either additively or by semitone scaling; the default semitone mapping is

$b_i = (x_i, y_i, x_i', y_i')$1

Earcons replace verbose filler such as “left parenthesis” and “end of fraction,” while IntuitiveSpeak provides short structural labels and MathSpeak remains available through loopback buffers. The editor supports linear caret movement, equation mode, and spatial keyboard strategies in which key location on the physical keyboard maps to grid location in the expression. It was iteratively co-designed with one blind and one sighted researcher across weekly sessions from February to July 2024, and a formal user study is planned.

At the other end of the interaction spectrum, the differentially processed optimized collaborative rich-text editor focuses on synchronization architecture rather than content semantics (Jatana et al., 2024). It partitions a large document into doclets, mounts only the active doclet as a full TipTap/ProseMirror editor instance, renders the others as static HTML, and routes all collaboration through a single full-duplex WebSocket shared across states. Each doclet is a Y.Doc with CRDT text types, and every message carries routing metadata such as app_id, state_id, user_id, session_id, op_type, op_id, seq, and version information. Demultiplexing is handled by a SharedStateManager on the client and a state router on the server, with routing complexity described as $b_i = (x_i, y_i, x_i', y_i')$2 average using a hashmap keyed by (app_id, state_id). Because Yjs updates are commutative, associative, and idempotent, the multiplexed channel preserves eventual consistency despite reordering or duplication. Measured WebSocket message exchanges show more than 96% improvement in the access-time proxy. In idle state, 1 editor falls from 172 to 4 messages per 5 seconds, 2 editors from 181 to 5, and 4 editors from 210 to 7. In active typing state, 1 editor falls from 1239 to 28, 2 from 1638 to 35, and 4 from 2094 to 52 (Jatana et al., 2024).

6. Limitations, misconceptions, and broader significance

A recurring misconception is that SMART-Editor denotes only a visually guided generative editor. The surveyed systems contradict that interpretation. FSolidM is a graphical and code-synchronized secure smart-contract editor; StereoMath is an audio-first equation editor; the collaborative rich-text system is a transport and state-tracking architecture; and the 2025 SMART-Editor framework is a multi-agent planner–critic–optimizer system for compositional design editing (Mavridou et al., 2018, Ge et al., 2 Jan 2025, Jatana et al., 2024, Mondal et al., 30 Jul 2025). Another misconception is that these systems are primarily local editors. The core motivation of the 2025 framework is precisely that insertions, deletions, replacements, and reordering operations trigger cascading global changes. Shap-Editor and SmartFreeEdit likewise emphasize preservation of identity, structure, geometry, lighting, and long-range consistency rather than isolated patch manipulation (Chen et al., 2023, Sun et al., 17 Apr 2025).

Their limitations are equally structured. SMART-Editor assumes reward signals that are sufficiently reliable, focuses on single-turn editing, and incurs inference-time overhead in Reward-Refine; complex constraints can still yield minor overlaps or misalignments, and deep narrative edits may plateau (Mondal et al., 30 Jul 2025). FSolidM is grounded in formal semantics but does not provide machine-checked correctness proofs, and its plugins cover only locking, ordering, timing, and access control (Mavridou et al., 2018). Shap-Editor is bounded by the quality of the Shap-E latent space and the 2D priors used for distillation; strongly out-of-distribution assets and complex topology changes remain difficult, and fully open-ended instruction following is described as requiring much larger-scale training (Chen et al., 2023). SmartFreeEdit remains vulnerable to ambiguous language, heavy occlusions, fine details, extreme lighting changes, and segmentation error propagation (Sun et al., 17 Apr 2025). StereoMath identifies audio clutter in very large expressions, the need for haptics or 3D audio for diagrams and geometric content, and the absence of voice and gesture input in the implemented system (Ge et al., 2 Jan 2025). The collaborative rich-text design reduces connection overhead but inherits head-of-line blocking risks from a single TCP stream, requires fairness scheduling across states, and depends on strict server-side authorization to prevent cross-state leakage (Jatana et al., 2024).

Taken together, these systems suggest that SMART-Editor is most productively understood as a theory of constrained editing: edits are mediated by a state space that can be reasoned over before execution, whether that state space is symbolic, geometric, latent, auditory-spatial, or CRDT-based. In that sense, the defining achievement of SMART-Editor is not merely automation, but the replacement of ad hoc edit application with representation-aware transformation that keeps semantics, structure, security, or synchronization intact across complex edits (Mondal et al., 30 Jul 2025, Mavridou et al., 2018, Chen et al., 2023, Ge et al., 2 Jan 2025, Sun et al., 17 Apr 2025, Jatana et al., 2024).