Proteus: Shapeshifting Desktop Visualizations for Mobile via Multi-level Intelligent Adaptation

Abstract: With the rise of mobile-first consumption, users increasingly engage with data visualizations on mobile devices. However, the vast majority of existing visualizations are originally authored for desktop environments. Due to significant differences in viewport size and interaction paradigms, directly scaling desktop charts often results in illegible text, information loss, and interaction failures. To bridge this gap, we propose an automated framework to adapt desktop-based visualizations for mobile screens. By systematically categorizing the operations involved in the adaptation process, we establish a multi-level design space. This space defines evolution rules spanning from the global topology level, through the reference frame level, down to the visual elements level. Guided by this theoretical framework, we developed Proteus, a LLM-driven multi-agent system that automatically parses online visualizations, predicts optimal transformation strategies within the design space, and generates equivalent, highly readable visualizations for mobile devices. Case studies and an in-depth user study with 12 participants demonstrate the effectiveness and usability of Proteus.

• The paper introduces a multi-level semantic adaptation framework that automatically transforms desktop visualizations for mobile use.
• It leverages a novel LLM-driven multi-agent system to reconfigure global layout, contextual references, and individual visual elements, ensuring data fidelity and usability.
• Empirical evaluations demonstrate that Proteus outperforms baseline methods in rendering success, perceptual readability, and interactive aesthetics across diverse chart types.

Proteus: Multi-Level Intelligent Adaptation for Mobile Visualization

Motivation and Problem Statement

The migration of interactive data visualizations from desktop to mobile devices is hindered by fundamental differences in viewport geometries, interaction modalities, and pixel density constraints. Traditional desktop visualizations rely on landscape orientation, high-resolution displays, and precise mouse-based interactions, resulting in dense layouts and rich context. Direct scaling of such visualizations to mobile environments causes critical failures, including illegible text, occluded marks, and interaction incompatibility. Previous responsive visualization techniques primarily utilize geometric heuristics and flat operator taxonomies, often sacrificing semantic fidelity and degrading cognitive usability in complex cases.

Multi-Level Design Space

Proteus introduces a hierarchical design space for mobile adaptation, operationalized as three layers:

• Global Topology: Macro-level structural reconfiguration, including axis transposition, layout serialization, and grid reflow, addressing at-a-glance spatial constraints (e.g., reorienting wide dashboards into vertical stacks for mobile).
• Reference Frame: Contextual adaptation of axes, legends, and scales, balancing quantitative accuracy and legibility (tick decimation, label rotation, legend repositioning, viewport constriction).
• Visual Elements: Micro-level refinements of marks and textual annotations, ensuring perceptual scalability and semantic equivalence (semantic abbreviation, label externalization, text wrapping, mark rescaling, data sampling).

This layered structure enables constraint propagation: higher-level structural decisions dictate feasible adaptations downstream, and element-level requirements can initiate topological changes upstream.

LLM-Driven Multi-Agent System

Proteus operationalizes semantic adaptation via a multi-agent system built on LLMs. Each agent encapsulates a specialized role:

• Semantic Parser: Multimodal analysis combining DOM/SVG structure and raster perception for precise topology deconstruction and segmentation.
• Data Extractor: Reverse-engineering of geometric and textual encoding from source code, outputting structured JSON data and mapping visuals to data semantics.
• Design Planner: Hierarchical decision-making referencing design requirements and action space, producing explicit transformation plans grounded in the multi-level framework.
• Frontend Engineer: Implementation in Next.js/TypeScript/TailwindCSS/Recharts, integrating prescribed structural adaptations, legend and interaction mechanisms.
• Visual Critic: Automated multi-criteria evaluation (data fidelity, plan adherence, text readability, aesthetics) with vision-capable LLMs, orchestrating iterative refinement cycles based on detected deficiencies.

This agentic workflow supports robust semantic re-authoring, iterative quality control, and automation across real-world heterogeneous visualization toolchains.

Case Studies

Proteus was validated on a curated set of real-world visualization examples spanning line charts, bar charts, maps, and scatter plots. Diverse scenarios demonstrate:

• Preservation and reorganization of horizontal scatterplot layouts into vertical stacks for enhanced legibility (without information loss).
• Conversion of dense in-plot annotations and controls into externalized, scrollable panels on mobile.
• Reimplementation of desktop-centric interaction paradigms (e.g., hover-based index adjustments) as mobile-native controls (range sliders, touch tooltips).
• Maintenance of cross-view coordination for multi-view visualizations by integrating flexible filtering widgets and range sliders contextualized with their associated views.

  Figure 1: Representative desktop visualizations and their mobile-adapted counterparts generated by Proteus, highlighting semantic, interaction, and layout transformations.

Empirical Evaluation

A controlled user study was conducted with 12 visualization experts, using a benchmark of 67 diverse desktop charts. Proteus was compared to a strong multi-agent LLM baseline (lacking hierarchical design space knowledge). Five evaluation dimensions were measured:

• D0: Execution completeness (binary rendering success)
• D1: Data fidelity (1–7 Likert)
• D2: Perceptual readability (1–7 Likert)
• D3: Interaction reasonableness (1–7 Likert)
• D4: Visual aesthetics (1–7 Likert)

Statistical analysis (Wilcoxon signed-rank test) revealed:

• Proteus achieved a 91.8% render success rate versus 87.8% for baseline.
• Proteus significantly outperformed the baseline in all qualitative dimensions ($p < 0.05$ for fidelity/readability; $p < 0.001$ for aesthetics and interaction).

  Figure 2: User study results comparing Proteus and the baseline; Proteus demonstrates statistically significant advantages in rendering, fidelity, readability, interaction, and aesthetics.

Case-wise qualitative assessment confirms Proteus's superior preservation of data semantics and functional layout structuring, particularly in complex, interactive scenarios.

Figure 3: Comparative examples showing that Proteus maintains accurate data mapping and readable layouts, whereas the baseline suffers from value distortion and suboptimal interactions.

Style and Preference Preservation

Proteus incorporates actionable constraints for style preservation. When requested, adaptation under such constraints maintains original color schemes and stylistic elements, while still achieving mobile usability.

Figure 4: Style preservation: adapted mobile visualization retains the source color scheme under explicit constraints.

Discussion, Implications, and Future Directions

Proteus’s multi-level adaptation paradigm provides a robust foundation for automated, semantic visualization consumption on mobile devices. Integration of LLM-driven multi-agent workflows enables robust cross-toolchain migration and responsive refinement. The introduced design space addresses semantic fidelity, perceptual scalability, and interaction redesign in a principled, hierarchical manner.

Remaining challenges include adaptation of highly bespoke or artistic visualizations (complex SVG/DOM scene graphs, custom spatial logics) and extraction from static raster charts (requiring chart-to-code vision models). Future research directions entail extending Proteus to raster-based inputs, broader end-user customization of adaptation strategies, task-oriented analytic equivalence evaluation, and fine-grained style preference mechanisms.

Conclusion

Proteus advances automated semantic adaptation of desktop visualizations for mobile consumption through a hierarchical design framework and LLM-driven, agentic multi-stage workflows. Empirical evaluation evidences robust improvements in rendering success, semantic fidelity, readability, interaction utility, and visual aesthetics compared to strong baselines. The multi-level propagation of design constraints is critical for practical and theoretically coherent adaptation. Proteus sets the stage for further research on pervasive, zero-intervention visualization consumption, adaptive stylistic preservation, and generalization to raster-only chart scenarios.

(2604.23299)