RiverONE: Generating Knowledge-Intensive VLM by Simulated Quantum Machines

Abstract: Quantum computing provides a powerful paradigm for representing and transforming high-dimensional information through superposition, entanglement, and measurement-induced nonlinear features. While current quantum hardware is not yet practical for direct large-scale vision-LLM (VLM) inference, simulated quantum computation can be used during model construction to generate structured parameters for compact classical AI systems. We build RiverONE, a lightweight vision-LLM for quantum calibration plot understanding, using simulated quantum computation. It employs a specialized visual encoder and an InternVL-based language backbone. To compensate for compression-induced information loss, we introduce quantum-generated parameters, which are materialized as classical tensors after training. This allows RiverONE to run entirely on classical GPUs at inference time, with no quantum hardware or runtime quantum simulation. With approximately 1.9 billion parameters, RiverONE achieves at least 95\% of the performance of NVIDIA Ising Calibration 1 on quantum calibration plot understanding tasks while using less than 10\% of its parameter count. These results suggest that simulated quantum computation can serve as a practical construction-stage mechanism for building lightweight, knowledge-intensive scientific VLMs. Our code is available at https://github.com/THeWakeSystems/RiverOne.

• The paper presents a novel VLM architecture that integrates simulated quantum parameter generation to offset compression losses, achieving 95% calibration accuracy.
• It employs a specialized visual encoder with MiniViT block sharing and quantum-inspired additive codebook quantization for efficient multimodal reasoning.
• The approach reduces resource demands by enabling deployment on consumer-grade GPUs, avoiding quantum hardware overhead during inference.

RiverONE: Generating Knowledge-Intensive VLMs via Simulated Quantum Machines

Architectural Overview

The RiverONE model is designed for quantum calibration plot understanding, integrating multimodal vision-language reasoning into a compact and efficient framework. Its architecture includes a domain-specialized ISingViT-800M visual encoder, pixel unshuffle and cross-layer concatenation, an MLP projector, and a language backbone based on InternVL-4B. Visual information is processed through the encoder, enhanced via unshuffle and concatenation, projected into the LLM's embedding space, and fused with tokenized chat messages for end-to-end reasoning.

Figure 1: RiverONE processes quantum calibration plot images with a specialized visual encoder, pixel operations, and an MLP projector to combine visual and textual signals within InternVL-4B.

Model Compression and Quantum-Generated Parameters

Compression is performed on both visual and language modules. The visual encoder employs MiniViT-style block sharing to reduce redundant computation, at the cost of layer-specific attention diversity. This compression is compensated by Quantum-Generated Parameter (QGP), which introduces simulated quantum variational circuit-based nonlinear weight transformations per layer.

The language backbone undergoes additive codebook quantization (AQLM), a quantum-inspired method that structures weight matrices as superpositions of codebook components. This analogy to quantum state superposition allows for tunable approximation quality and efficient use of memory, maintaining domain-specific reasoning in a 1.9B parameter budget.

Figure 2: Compression in RiverONE leverages block sharing in the visual encoder and quantum-inspired codebook quantization for the LLM, with QGP providing compensation.

Quantum-Generated Parameter Pipeline

QGP is strictly a construction-stage mechanism: compressed shared weight matrices are encoded as quantum states via amplitude encoding, processed through parameterized variational quantum circuits (VQCs), and materialized as classical tensors following linear readout. Quantum entanglement in the VQC allows for layer-adapted weights that capture cross-feature correlations lost in classical adapters.

Figure 3: QGP pipeline generates layer-specific compensated weights through simulated quantum processing, materialized into classical tensors for deployment.

The resulting weights are inserted into selected attention blocks within the visual encoder and used during end-to-end training. After training, inference is entirely classical, with quantum simulation never required at runtime.

Progressive Compression and Training Protocol

RiverONE is trained via a sequential protocol:

• Supervised Fine-Tuning (SFT): Two stages—first, in-context demonstration to build calibration reasoning, followed by zero-shot fine-tuning to force internalization of domain knowledge.
• Mixed Preference Optimization (MPO): Offline preference-based reinforcement learning, contrasting accepted and rejected responses to fine-tune visual grounding, fit reliability, and scientific reasoning.

The combination of SFT and MPO maximizes calibration-specific knowledge retention pre-compression, with QGP restoring capacity lost to visual block sharing.

Empirical Results

Comparative evaluations using QCalEval show that RiverONE-1.9B achieves at least 95% of the performance of NVIDIA Ising Calibration 1 while using fewer than 10% of its parameters. Ablation studies demonstrate that MiniViT compression incurs performance degradation in categories requiring layer-specific attention, but QGP compensation restores nearly all lost accuracy. Language compression via AQLM is critical for memory efficiency and scales effectively with minimal loss. RiverONE requires only a single consumer-grade GPU for deployment compared to server-class requirements for baselines.

Theoretical and Practical Implications

RiverONE empirically validates the hypothesis that quantum-inspired parameter generation can serve as a practical mechanism for constructing lightweight, knowledge-intensive scientific VLMs. By restricting quantum computation to the training phase, it sidesteps quantum hardware readiness and delivers models fully deployable on classical infrastructure. The QGP pipeline enables flexible nonlinear compensation critical for scientific domains, where compression-induced representational loss can have significant impact on inference quality.

The architecture's reliance on quantum-inspired constructs aligns with recent advances in hybrid quantum-classical ML (Liu et al., 2024), but pushes the paradigm towards purely classical inference. This separation of quantum and classical roles allows practitioners to exploit quantum expressive power without being bottlenecked by hardware or simulation overhead.

Future Directions

Anticipated developments include:

• Expansion of calibration-specific training datasets to further improve domain coverage.
• Deeper analysis of compensatory gains: quantifying which encoder layers benefit most from QGP and exploring stronger classical generator baselines.
• Application of materialized quantum-derived parameter generation to other scientific VLMs beyond quantum calibration, including microscopy, spectroscopy, and time-series diagnostics.
• Integrating structured quantum priors into visual encoder architectures for other scientific domains.

Conclusion

RiverONE establishes a framework for leveraging simulated quantum computation as a construction-stage tool for compact scientific VLMs. It demonstrates that quantum-inspired parameter generation can effectively compensate for capacity lost to model compression, enabling the deployment of knowledge-intensive models on classical hardware in scientific settings. This approach broadens the scope of practical, domain-aware AI solutions and suggests significant potential for further cross-pollination between quantum computing paradigms and classical model architectures.