VizGenie: Self-Improving Scientific Visualization

• VizGenie is a self-improving, domain-aware system for HPC-scale 3D visualization that integrates static modules with dynamic, LLM-driven code generation.
• The framework employs large language models to orchestrate code synthesis, validation, and iterative refinement, streamlining complex volumetric rendering tasks.
• It combines image-based visual question answering with persistent provenance tracking to ensure reproducible, feature-centric volumetric analysis.

VizGenie is a self-improving, domain-aware workflow for scientific visualization that was introduced as an agentic framework for HPC-scale, 3D volume visualization with LLMs, dynamically generated visualization modules, image-based visual question answering, Retrieval-Augmented Generation, and persistent provenance (Biswas et al., 18 Jul 2025). In adjacent design literature, the name also appears as a blueprint for automatic visualization and infographic generation inspired by LIDA’s multi-stage pipeline of data summarization, goal enumeration, visualization generation, and stylization (Dibia, 2023). Across these usages, VizGenie denotes a class of systems that reduce cognitive overhead in visualization by coupling natural-language interaction with automated synthesis, validation, and iterative refinement.

1. Problem domain and scope

VizGenie was formulated against a specific bottleneck in scientific visualization: users working with volumetric data and multi-field simulation outputs must know which algorithms to apply, how to tune parameters, and how to translate domain language into concrete operations such as volume rendering, isosurfacing, and slicing (Biswas et al., 18 Jul 2025). The motivating example is feature-centric language such as “visualize the skull,” which implies semantic feature localization rather than a low-level pipeline specification.

The 2025 formulation is explicitly positioned as different from many existing LLM-based systems in two ways. First, it is not a single-pass code generator that expects repeated human repair. Second, it is not primarily a 2D information-visualization system; it targets HPC-scale, 3D scientific visualization with volumetric rendering, feature highlighting, and follow-up interrogation of rendered outputs (Biswas et al., 18 Jul 2025).

A recurrent source of confusion is nomenclature. A separate system named VizGen is documented as a multi-agent architecture for natural-language-driven SQL analytics and graph generation, and that paper explicitly states that it does not mention “VizGenie” (Fernando et al., 26 Sep 2025). By contrast, VizGenie in the scientific-visualization sense centers on VTK-based rendering, domain-aware VQA, and persistent capability expansion rather than SQL translation and dashboard-style chart recommendation.

2. Architectural composition

The core VizGenie architecture combines stable hand-engineered tools with dynamically synthesized modules. Pre-existing modules cover threshold-based filtering, slice extraction, and statistical analysis. Examples of tools selected through zero-shot ReAct-style descriptions include FilterRuns, VisualizeRun, AnalyzeRuns, and VisualizeHistogram. When a request exceeds the baseline toolset, VizGenie invokes an LLM-based CodeGenerator to synthesize new VTK Python scripts, validates them, and stores successful modules in a SQLite cache for later reuse (Biswas et al., 18 Jul 2025).

A single LLM agent performs orchestration and tool selection. The paper specifies GPT-4o in deterministic mode for orchestration and general question answering, o3-mini with temperature $=1$ for creative code generation, and GPT-4o-mini in deterministic mode for targeted code refinement. The interface is implemented as a PyQt5 chat application with panels for chat, agent reasoning, 2D image display, and interactive 3D rendering through QVTKRenderWindowInteractor (Biswas et al., 18 Jul 2025).

Component	Role	Implementation notes
Pre-existing modules	Foundation tasks	threshold-based filtering, slice extraction, statistical analysis
Dynamically generated modules	Capability expansion	VTK Python scripts synthesized on demand and cached in SQLite
Orchestrator	Planning and tool use	single LLM-agent with zero-shot-react-description
Vision subsystem	Captioning and VQA	locally fine-tuned Llama-3.2-Vision (11B) via LoRA
Retrieval and memory	Context and reproducibility	RAG over historical interactions and local documents

The resulting dataflow is sequential and stateful. A natural-language query is routed either to an existing tool or to the code-generation pathway; if generated code fails validation, ModifyGeneratedCode attempts iterative repair; rendered outputs can then be re-ingested by the VQA subsystem for follow-up questions; successful runs are recorded with prompts, parameters, tools used, and validation status (Biswas et al., 18 Jul 2025). This architecture makes module synthesis part of normal operation rather than a one-off fallback.

3. Feature-centric interpretation and volumetric operations

VizGenie’s feature-centric behavior depends on image-based interpretation rather than only text parsing. For domain-aware captioning and VQA over rendered images and screenshots, the system uses a locally fine-tuned Llama-3.2-Vision (11B) model with LoRA adapters of the form $W' = W + AB$, where $A \in R^{d \times r}$, $B \in R^{r \times d}$, and $r \ll d$ (Biswas et al., 18 Jul 2025). The model is fine-tuned on LANL-generated and curated images for Deep Water Impact to improve domain vocabulary and stability.

The operational primitives are standard scientific-visualization constructs. Thresholding uses a binary mask $M(x) = [f(x) \ge T]$, and range filtering uses $M(x) = [T_1 \le f(x) \le T_2]$. Statistics operate on the restricted field, including mean, variance, and a pooled-variance $t$-test with Cohen’s $d$ when two-sample comparisons are requested. Slice extraction is modeled by plane sampling on $\Pi: ax + by + cz + d = 0$, with axis-aligned slices as 1-hot choices of $W' = W + AB$0 and $W' = W + AB$1, implemented with VTK image reslicers and extractors (Biswas et al., 18 Jul 2025).

For volumetric rendering, VizGenie relies on the standard emission–absorption model

$W' = W + AB$2

together with discrete front-to-back compositing over transfer-function-derived color and opacity samples. The paper presents on-demand module generation for volume rendering with custom transfer functions, isosurface extraction, feature highlighting, outlines, camera control, and colormap or opacity customization (Biswas et al., 18 Jul 2025).

A distinctive mechanism is its isovalue recommendation strategy. VizGenie generates screenshots across camera angles and candidate isovalues, captions them with the fine-tuned vision model, and uses LLM-based retrieval and ranking over the caption database to map phrases such as “skull” or “teeth” to appropriate parameter values. In the headsq example, predicted optimal isovalues moved closer to expert ranges after self-improvement: skull shifted from $W' = W + AB$3 to $W' = W + AB$4 against an expert range of $W' = W + AB$5–$W' = W + AB$6, and teeth shifted from $W' = W + AB$7 to $W' = W + AB$8 against an expert range of $W' = W + AB$9–$A \in R^{d \times r}$0 (Biswas et al., 18 Jul 2025).

4. Dynamic code synthesis, validation, and self-refinement

Generated modules follow explicit integration constraints. The agent assembles a “Must implement” specification, including requirements such as valid Python code using VTK and a standardized entry point update_vtk_scene(renderer). User overrides from chat are injected into this specification, and successful modules are stored with metadata such as prompt, dataset path, and parameters (Biswas et al., 18 Jul 2025).

The validation pathway is central rather than auxiliary. Each generated module is executed in a sandbox, runtime logs are captured, and iterative refinement through ModifyGeneratedCode continues until validation succeeds or an iteration cap is reached. If repeated repair fails, VizGenie degrades to a simpler visualization or another pre-existing tool (Biswas et al., 18 Jul 2025). This workflow is designed to increase robustness while preserving the extensibility benefits of code synthesis.

The self-refining loop is asynchronous. The system expands its visual database by generating more isovalues and camera angles, regenerates captions with the fine-tuned vision model, and improves feature-to-parameter mappings over time. On the headsq dataset, vocabulary size increased from $A \in R^{d \times r}$1 with $A \in R^{d \times r}$2 images $A \in R^{d \times r}$3 isovalues $A \in R^{d \times r}$4 angles$A \in R^{d \times r}$5 to $A \in R^{d \times r}$6 with $A \in R^{d \times r}$7 images $A \in R^{d \times r}$8, and mean pairwise semantic similarity increased from $A \in R^{d \times r}$9 to $B \in R^{r \times d}$0 (Biswas et al., 18 Jul 2025).

The same section of the paper emphasizes that capability growth is persistent. Validated code is not merely executed once; it becomes part of the cached toolset. A plausible implication is that VizGenie treats generated visualization logic as accumulated infrastructure rather than ephemeral output.

5. Retrieval, provenance, interface, and broader pipeline interpretations

VizGenie records extensive provenance. Stored metadata includes prompts and agent reasoning traces, tool invocations, parameters such as data path and isovalue, runtime logs, generated code with validation status, and image-caption tuples containing dataset identifier, isovalue, camera angle, and caption. The paper gives conceptual schemas for modules, images, and interactions, and notes that reloading a prior prompt, dataset, and cached module can reproduce a result through the same update_vtk_scene entry point (Biswas et al., 18 Jul 2025).

Its RAG subsystem is described as lightweight and retrieves from historical chat logs and local domain documents such as contest PDFs. The retrieval layer improves answers to questions like “what fields are in the grid?” and supplies parameter context; the paper explicitly states that chunking strategies, embedding models, vector store type, top-$B \in R^{r \times d}$1, and reranking are not specified (Biswas et al., 18 Jul 2025). This bounded description is important: VizGenie uses retrieval for grounding, but its RAG layer is not presented as a fully elaborated retrieval benchmark.

The interface couples transparency with direct manipulation. The PyQt5 GUI exposes chat, reasoning traces, 2D images, and interactive 3D rendering, while the reasoning panel surfaces errors, warnings, and validation outcomes (Biswas et al., 18 Jul 2025). In the broader design literature associated with the VizGenie name, this transparency aligns with a four-module pipeline derived from LIDA: SUMMARIZER, GOAL EXPLORER, VISGENERATOR, and INFOGRAPHER. That blueprint frames automatic visualization as a multi-stage generation problem with data summarization, goal enumeration, executable code generation with execution-and-filtering, and text-conditioned image-to-image stylization for data-faithful infographics (Dibia, 2023).

This suggests two related but non-identical meanings of VizGenie in current literature. One denotes a domain-specific scientific-visualization framework for VTK and volumetric data (Biswas et al., 18 Jul 2025). The other denotes a generalized automatic-visualization architecture inspired by LIDA’s grammar-agnostic, multi-stage orchestration (Dibia, 2023).

6. Empirical results, limitations, and relation to adjacent research

The quantitative evaluation of the scientific VizGenie centers on four datasets: CT headsq.vti, Hurricane Isabel, Ionization Front, and Deep Water Impact. Baseline tools such as slicing and histograms complete in approximately $B \in R^{r \times d}$2–$B \in R^{r \times d}$3 seconds on a Macbook Pro M3, whereas code generation and refinement take approximately $B \in R^{r \times d}$4–$B \in R^{r \times d}$5 seconds. For captioning and VQA on Deep Water Impact, inter-caption semantic similarity across camera angles improved from $B \in R^{r \times d}$6 to $B \in R^{r \times d}$7 after fine-tuning, with a Mann–Whitney $B \in R^{r \times d}$8 test reporting $B \in R^{r \times d}$9. Vocabulary became more domain-focused: total size decreased from $r \ll d$0 to $r \ll d$1, while terms such as “crater,” “asteroid,” and “impact” increased sharply. Human-in-the-loop accuracy rose to $r \ll d$2 true statements post-tuning, versus effectively $r \ll d$3 pre-tuning; LLM-as-judge scores also improved, with GPT-4o rating fine-tuned captions at $r \ll d$4 versus $r \ll d$5 out-of-the-box, and o4-mini-high scoring $r \ll d$6 versus $r \ll d$7 (Biswas et al., 18 Jul 2025).

The paper is explicit about limitations. Multiple LLM calls introduce latency; smaller models can be faster but may reduce validity. VQA may misinterpret ambiguous or amorphous features. Automated execution checks guarantee runnable code but do not guarantee scientific correctness. The RAG mechanism remains lightweight, and provenance does not yet standardize code hashes or containerized execution (Biswas et al., 18 Jul 2025). These caveats constrain how far validation should be interpreted: robustness of software execution is not equivalent to correctness of scientific interpretation.

The surrounding literature clarifies how VizGenie sits within a broader research landscape. “Visualization by Example” decomposes synthesis into a data-wrangling DSL and a visualization DSL, with inverse semantics and bi-directional program analysis to infer full visualizations from small sketches (Wang et al., 2019). LIDA addresses grammar-agnostic automatic visualization and infographic generation through staged summarization, goal exploration, code synthesis, execution filtering, and stylization, reporting a visualization error rate of $r \ll d$8 in its initial benchmarks (Dibia, 2023). VizAI studies visualization selection as a generative–discriminative problem, choosing the chart whose inferred statistics best match the data’s true statistics (Vij et al., 2021). VizRec adds statistical guardrails for visual recommendation, using classical tests and VC-dimension-based uniform guarantees to control the probability of misleading recommendations under adaptive exploration (Stefani et al., 2018).

Taken together, these systems indicate that “VizGenie” is best understood not only as a single platform but also as a convergence point for several lines of work: automated visualization synthesis, feature-centric scientific rendering, LLM-orchestrated module generation, retrieval-grounded interaction, statistical reliability, and iterative self-improvement. In the strict sense of the 2025 paper, however, VizGenie remains a specialized framework for reproducible, self-refining scientific visualization in HPC settings rather than a generic natural-language charting assistant (Biswas et al., 18 Jul 2025).