BrushData: Digital Brush Operation Data

• BrushData is a comprehensive specification for digital brush operations that encodes geometric, operational, and contextual parameters for interactive systems.
• It leverages real-time sampling and ergonomic parameterization to support methodologies in immersive 3D drawing and diffusion-based image editing.
• BrushData underpins advanced workflows in spatial drawing, high-dimensional data visualization, and scalable image synthesis, enabling precise and intuitive user interactions.

BrushData broadly refers to the specification, representation, and manipulation of data associated with digital brush operations in interactive systems. The term appears in multiple research contexts: it may denote the encoded data and parameters that define physical or virtual brushes (for drawing, editing, or synthesis), the user’s input trajectory and controller signals, real-time manipulation settings, or metadata that guides image and surface manipulation workflows. BrushData is a key element in enabling intuitive, accurate, and semantically meaningful interactions in digital content creation and analysis, with applications spanning 3D spatial drawing, image editing with diffusion models, multidimensional data visualization, and infinite-resolution synthesis.

1. Formalization and Representation of BrushData

BrushData typically encodes the geometric, operational, and contextual properties of a digital brush action. In 3D spatial drawing interfaces such as StripBrush (Rosales et al., 2021), BrushData includes:

• The sampled positions of the input device (e.g., controller, stylus) over time ($\mathbf{p}_n$)
• Orientation information (e.g., "side" axis, "up" axis, or full six-DoF pose)
• Operational states (e.g., draw/trigger engagement)
• Per-stroke parameters such as ribbon width

In advanced image editing frameworks built on diffusion models (e.g., Layered Diffusion Brushes (Gholami et al., 1 May 2024)), BrushData also includes:

• Spatial mask information specifying target regions
• Prompt text accompanying each brush stroke
• Adjustable parameters: brush strength ($\alpha$), denoise step count ($n$), random seed values (for noise generation), and layer order
• Intermediate latent representations ($Z_{(r)}, Z_{l_0}$) necessary for real-time local manipulation

In multidimensional projection interfaces (e.g., distortion-aware brushing (Jeon et al., 2022)), BrushData comprises:

• Indices of selected points within the projection
• Corresponding high-dimensional coordinates
• Interaction context (timing, brush region, current projection state)

A unifying feature is that BrushData is both interactive (constructed and modulated in real-time) and parameterized (enabling replay, auditing, or advanced manipulation downstream).

2. Methodologies for BrushData in Immersive 3D and Spatial Drawing

In immersive VR drawing, BrushData captures continuous control over both position and orientation. StripBrush (Rosales et al., 2021) introduced a departure from conventional "normal brush" operation. Instead of requiring strict alignment of the controller’s "up" axis with the desired surface normal (which produces an "overconstrained" input and ergonomic strain), the StripBrush interface defines BrushData so that the side axis of the controller directly encodes the ruled surface’s orientation. Each sampled position $\mathbf{p}_n$ along with the side direction collectively specify ribbon boundaries:

• Ruling endpoints are placed at a fixed offset (half ribbon width) from $\mathbf{p}_n$ along the instantaneous side direction.
• The system estimates the stroke’s final normal from the orientation of the rulings instead of enforcing an explicit constraint.

This parameterization enables ergonomic, accurate, and physically less demanding spatial sketching, especially for surfaces with widely varying curvature.

3. BrushData in Diffusion-Based Image Editing

In image editing frameworks utilizing denoising diffusion models, BrushData encapsulates both spatial and semantic controls (Gholami et al., 1 May 2024):

• For each edit, a mask defines the region to be modified; a prompt provides high-level guidance (e.g., textual description).
• Intermediate latents ($Z_{(r)}, Z_{l_0}$) are cached at specific diffusion steps.
• The edit is realized by introducing scaled random noise into the masked region and propagating the change via partial denoising steps.

The adjustment strength is governed by a formula (editor’s term: "Brush Strength Law"):

$$\alpha = \sqrt{\left| \frac{(\alpha^*/100) \cdot (\sigma - 2 \cdot \mathrm{Cov}(Z_{n_k}, x_0')/\mathrm{Var}(Z_{n_k}))}{\sum[\text{mask pixels}]/W} \right|}$$

where $\alpha^*$ is user-configured, $\sigma$ is a fixed variance parameter, and $W$ is the latent width.

BrushData here is crucial for enabling real-time, iterative editing with layered controllability. Each "edit layer" is fully characterized by its own BrushData tuple: (mask, prompt, local seeds, blend/caching state, control parameters).

4. BrushData and Reliable Selection in High-Dimensional Projections

In multidimensional data visualization, specifically when using multidimensional projections (MDPs), BrushData functions as the operational record of the user's brushing interaction (Jeon et al., 2022). Due to projection-induced distortions, simple geometric regions in 2D do not reliably correspond to clusters in the original space. Distortion-aware brushing dynamically corrects for this:

• BrushData encodes both the selection in the projection and the index mapping to the high-dimensional space.
• Using transformation functions $f(\|\mathbf{x}_i - \mathbf{x}_b\|)$, the visible positions $p_i$ in the projection are iteratively updated:

$$p_i^{\text{new}} = p_i + f(\|\mathbf{x}_i - \mathbf{x}_b\|) \cdot (p_b - p_i)$$

This process continuously updates the projection to maintain a brush-induced local correspondence with clusters in the original space, making BrushData foundational to distortion correction–driven interaction.

5. High-Resolution Synthesis: Function-Space BrushData

In large-scale, resolution-agnostic image synthesis, as in $\infty$-Brush (Le et al., 20 Jul 2024), BrushData can generalize to a set of function-space coordinates, auxiliary condition signals, and intermediate state representations:

• Each brush interaction can be thought of as a set of conditioning vectors $e$, possibly combined with sampled query points or coordinate sets over a continuous image domain.
• The cross-attention neural operator consumes this BrushData to inject spatially- and semantically-resolved guidance into the diffusion process.

This enables synthesis and manipulation at arbitrary scale, essential in domains such as histopathology or satellite imagery where classical grid-aligned BrushData is computationally infeasible.

6. Comparative Table of BrushData Attributes

Context	BrushData Elements	Key Role
3D Spatial Drawing	Positions, orientations, ribbon parameters	Geometry and ergonomics of brush strokes
Diffusion Image Editing	Mask, prompt, latent cache, control parameters	Localized, real-time, prompt-driven edits
MDP Visualization	Selection indices, coordinate mappings, timing	Robust, accurate cluster selection
Infinite-Dim Synthesis	Condition vectors, sampled coordinates, auxiliary	Arbitrary-scale, globally consistent edits

7. Applications, Implications, and Future Research Directions

BrushData underpins a wide spectrum of advanced digital content workflows:

• In VR and 3D design, it supportsmore ergonomic, accurate, and expressive user interactions.
• In contemporary AI-driven image editing, BrushData enables rapid, localized, and compositional edits while supporting high levels of semantic control, as well as layer management analogous to traditional raster graphics packages.
• In exploratory multidimensional data analysis, it improves cluster separability and interpretability through interactive, distortion-aware manipulation.
• In function-space models, it ensures scalability and enables control of very large images without a prohibitive computational footprint.

Future research will likely further refine the representation and manipulation of BrushData—optimizing ergonomic mapping, introducing adaptive multimodal (e.g., haptic) feedback, integrating with dynamic neural operator architectures, and supporting increasingly complex or high-dimensional editing tasks. Enhancements in real-time feedback and conditional control parameters remain essential for maximizing the utility and expressive power of BrushData-driven interfaces.