QuantumQA Dataset

• QuantumQA is a large-scale dataset of 92,749 QA pairs designed to enable scientifically valid reasoning in quantum mechanics.
• It employs a hybrid verification protocol that integrates deterministic scientific solvers with LLM-based semantic auditing to ensure both physical consistency and semantic correctness.
• The dataset supports diverse task formats—from short answers to complex problem solving—facilitating robust evaluation and reinforcement learning for advanced quantum models.

QuantumQA is a large-scale quantum mechanics question–answer (QA) dataset rigorously constructed to enable the training, evaluation, and alignment of LLMs for scientifically valid reasoning in quantum domains. Designed to address the scarcity of verifiable, high-quality resources in quantum mechanics, QuantumQA employs a task-adaptive generation strategy and a hybrid verification protocol that tightly integrates deterministic scientific solvers with LLM-based semantic assessment. The result is a dataset intended to guarantee both physical consistency and semantic correctness, supporting advanced reinforcement learning paradigms and robust model evaluation (Qu et al., 20 Apr 2026).

1. Dataset Composition and Topical Distribution

QuantumQA comprises 92,749 QA pairs spanning five principal task formats: Short Answer, Fill-in-the-Blank, True/False, Multiple Choice, and extended Problem Solving. The dataset covers broad thematic ground in quantum mechanics through multi-labeled categorization. The ten most frequent topics (with approximate frequency) are:

Topic	Portion
Particle in a Box	~12%
Hydrogen Atom	~10%
Spin Systems & Angular Momentum	~9%
Quantum Optics	~8%
Perturbation Theory	~7%
Density Matrices & Mixed States	~6%
Entanglement & Bell Inequalities	~6%
Time-Dependent Processes	~5%
Quantum Information Basics	~5%
Scattering & Potential Theory	~5%

Percentages sum to more than 100% due to multi-label assignment at the question level. Difficulty annotations are primarily based on concept depth and step complexity: ~45% undergraduate, ~35% graduate, and ~20% research-level (characterized by highly advanced reasoning or technical novelty). Data splits include approximately 95% for training (of which 70% is used for supervised fine-tuning), 0.1% for development, and 5% (4,675 examples) for held-out test evaluation (Qu et al., 20 Apr 2026).

2. Item Structure and Task Format

Each QuantumQA entry encapsulates a four-block format:

1. System Prompt: Defines the expert persona and context, e.g., “You are an expert quantum physicist…”
2. Instruction: Sets task-specific constraints, such as “Compute the energy eigenvalues…”
3. Question: The substantive problem statement, often including LaTeX-encoded mathematical expressions.
4. Answer: A model-generated or programmatically validated solution, sometimes including explicit Chain-of-Thought (CoT) reasoning.

QuantumQA supports both structured formats (multiple choice, true/false) and open-ended mathematical derivations, with distractors for deterministic tasks carefully synthesized or verified. Representative examples encompass both symbolic manipulations (e.g., determining the commutator of time-evolved ladder operators) and conceptual derivations (e.g., finding the hydrogen atom spectrum from the Schrödinger equation) (Qu et al., 20 Apr 2026).

3. Data Generation and Expansion Pipeline

The generation protocol proceeds through four phases:

1. Seed Initialization:
   • Textbook sources (e.g., Griffiths, Nielsen & Chuang) undergo OCR ingestion.
   • DeepSeek-V3 extracts foundational theorems, core definitions.
   • Redundancy reduced by semantic deduplication (cosine similarity threshold 0.85), leaving ~1,000 unique seed items.
2. Concept Decomposition:
   • Qwen3-Max performs hierarchical decomposition to extract granular sub-topics.
   • The same deduplication protocol maintains coverage without topic conflation.
3. QA Pair Synthesis:
   • An ensemble of LLMs (DeepSeek-V3, Qwen3-Max, ChatGPT-5) generates pairs according to task type:
     • For deterministic tasks, unique solutions and balanced distractors are synthesized.
     • Problem-solving questions follow a two-step pipeline generating both the question and annotated answer with a difficulty label.
4. Adaptive Chain-of-Thought (CoT) Injection:
   • For “Hard” or research-level tasks, the ensemble generates and tags detailed stepwise reasoning in answer blocks using CoT markup.

This framework supports scalable coverage and enables targeting of both breadth (multi-topic) and depth (highly detailed derivations) (Qu et al., 20 Apr 2026).

4. Hybrid Verification and Quality Control Protocol

QuantumQA employs a two-pronged verification regime:

• Automated Verification via Scientific Execution Suite (SES):
  • SES, built from SymPy and QuTiP scripts, programmatically checks:
  • Symbolic correctness (e.g., validation of algebraic manipulations or integral computations).
  • Physical constraints (e.g., normalization, boundary conditions, commutation relations, operator unitarity, positivity of density matrices).
  • Each test dimension (correctness $\mathit{Corr}$, physical validity $\mathit{Phys}$) yields a discrete pass (1), not applicable (0), or fail (−1) value.
• LLM-based Semantic Auditing:
  • An independent LLM evaluates instruction-following fidelity ($\mathit{Inst}$).
• Human-in-the-Loop Audit:
  • Spot checks on SES-passing batches by quantum domain experts.
  • Batches with >5% error are entirely discarded and regenerated.

Structured statistical audits further underpin quality, e.g., semantic and deterministic pass rates (Phys: 68.0%, Corr: 81.4%, Inst: 92.6% over $N=500$ trajectories) and human spot-check reduction in logical violation rates from 41% (supervised baseline) to 28% (post-SES filtering) (Qu et al., 20 Apr 2026).

5. Error Typology and Representativity

Error tracking distinguishes between purely deterministic failures (incorrect algebra), physical violations (unsound or non-physical solutions), and instruction-following lapses. An example of SES filtration: an answer with correct algebraic evaluation but a ground-state quantum number $n=0$ for a particle in a box is excluded as unphysical (violates the uncertainty principle). Both deterministic and semantic checks are cross-tabulated in confusion matrices for diagnostic transparency. Physical consistency is maintained to a tolerance of ≤0.1% relative error in computed energy levels, and all equations must be dimensionally homogeneous (Qu et al., 20 Apr 2026).

6. Evaluation, Fine-Tuning, and Use Cases

QuantumQA supports:

• Supervised Fine-Tuning (SFT): The 70% SFT subset trains models on both format, content, and distractor identification.
• RL with Verifiable Rewards (RLVR): Training protocols incorporate direct SES-based reward computation, with the Verification-Aware Reward Model (VRM) fusing SES and semantic evaluator signals.
• Benchmarking: The 5% test split is used for performance evaluation, commonly via an LLM-as-judge accuracy score (ACC_U, normalized 0–1).
• Curriculum and Progressive Training: Entries are indexed by topic and difficulty (undergraduate → research); suitable for targeted instructional design.
• Potential Extensions: Planned expansions include support for quantum circuit diagrams, prompts for open quantum systems, further domain generalization (chemistry, classical mechanics), and synthetic augmentation of CoT traces via advanced LLMs filtered for solution validity (Qu et al., 20 Apr 2026).

7. Comparison and Positioning Within the Quantum Data Ecosystem

QuantumQA is distinguished by its hybrid verification, rigorous filtering regime, and extensive topic coverage compared to other quantum datasets. Unlike QuantumBench (Minami et al., 30 Oct 2025), which targets undergraduate-to-early-graduate-level multiple-choice tasks with tight distractor curation, QuantumQA aims for broader depth, including research-level problems and open-ended tasks. In contrast to QDataset (Perrier et al., 2021), which provides quantum dynamics data for machine learning, and QuantumLLMInstruct (Kashani, 2024), which offers a large corpus of instruction–solution pairs with LLM-based verification, QuantumQA uniquely emphasizes scientific fidelity through deterministic/semantic fused verification. This approach is explicitly constructed to mitigate the insufficiency of preference-model alignment for reasoning in quantum mechanics, supporting research on trustworthy, physics-consistent LLM development (Qu et al., 20 Apr 2026).

---

QuantumQA thus represents a rigorously constructed quantum-mechanics reasoning benchmark that anchors LLM evaluation and alignment in reproducible, verifiable, and semantically consistent scientific standards. Its fusion of deterministic (SES) and soft (LLM) verification addresses both the risk of symbolic hallucination and the brittleness of purely rule-based curation, ensuring dataset integrity for advanced model development.