PennyCoder: Quantum Code Generation LLM

• PennyCoder is a domain-specific large language model for quantum code generation in the PennyLane framework, offering on-device, privacy-preserving solutions.
• It leverages parameter-efficient adaptation via LoRA and instruction tuning on a curated PennyLang dataset to enhance quantum programming and optimize performance.
• The model demonstrates improved accuracy with reduced hallucinations and efficient resource usage, supporting advanced quantum machine learning and reinforcement learning use cases.

PennyCoder is a domain-specific LLM system designed for high-fidelity quantum code generation in the PennyLane framework. It facilitates local, privacy-preserving, and efficient quantum programming assistance, particularly emphasizing on-device applicability and support for quantum machine learning and quantum reinforcement learning use-cases. PennyCoder integrates parameter-efficient LLM adaptation, a rigorously curated PennyLane-centric dataset, and evaluation methodologies tailored to the functional requirements of quantum software development (Basit et al., 4 Mar 2025, Basit et al., 25 Jul 2025).

1. Dataset Construction and Annotation

PennyCoder’s code generation capabilities stem from instruction–code corpora curated explicitly for PennyLane. The foundation, termed the PennyLang dataset, comprises 3,347 unique samples. The construction pipeline aggregates content from multiple sources:

• Community and Official Code: 1,952 samples were scraped from permissively licensed (MIT, Apache-2.0, BSD) GitHub repositories containing "pennylane as qml" imports. An additional 1,321 samples came from the official PennyLaneAI repository.
• Textbook Extraction: 21 rigorously hand-verified code samples were sourced from canonical texts (notably Schuld & Petruccione, Biamonte et al.), emphasizing coverage and annotation quality.
• Documentation Crawling: 53 samples were curated from tutorials and API reference sections of docs.pennylane.ai, extracting fenced Python for relevant context.

Each sample is formatted as NDJSON, with structured fields: a natural-language instruction, code, context, comment-to-line-index alignment, and feature tags (e.g., “qml.RX”, “measurement”). PEP 8 formatting via Black, hash-based duplicate removal, and stratified category tagging (gates, templates, measurement, optimization, data prep, error handling, math) ensure consistency and coverage. Randomized human verification (10% sample) calibrates congruence between instruction and code (Basit et al., 4 Mar 2025).

2. Model Architecture and Parameter-Efficient Adaptation

PennyCoder’s model backbone is the LLaMA 3.1-8B decoder-only Transformer, characterized by 32 layers, hidden size $d=4096$, 32 attention heads, and a vocabulary $\approx32$k tokens. To enable practical on-device deployment and circumvent the resource demands of full fine-tuning, PennyCoder applies Low-Rank Adaptation (LoRA) [Editor’s term: adapter tuning].

• LoRA Mechanism: For base weight matrix $W \in \mathbb{R}^{d\times d}$ in attention layers, adaptation is:

$W' = W + \Delta W,\quad \Delta W = AB$

where $A \in \mathbb{R}^{d\times r}$, $B \in \mathbb{R}^{r\times d}$, and $r = 8 \ll d$. Only $A, B$ are optimized; $W$ is frozen (Basit et al., 25 Jul 2025).

• Optimization Details: AdamW optimizer (weight decay 0.01), initialization $A,B \sim \mathcal{N}(0, 10^{-2})$, dropout on $\approx32$0, LoRA applied to query and value projections.
• Resource Implication: This yields $\approx32$10.2% parameter overhead, permitting LoRA adapter weights ($\approx32$230 MB) to be merged into LLaMA 3.1-8B during inference.

3. Instruction Tuning and Training Pipeline

Domain specificity is attained through instruction tuning with the PennyLang dataset. The training paradigm employs supervised learning on (instruction, code) pairs covering:

• Elementary circuits (Hadamard, CNOT, measurement)
• Variational quantum algorithms (VQE, QAOA)
• Quantum machine learning (QML), quantum reinforcement learning (QRL)
• Advanced constructs such as agent architectures and reward loops

Training configuration:

• Split: 90% training, 10% validation
• Hardware: NVIDIA A100 80GB
• Epochs: 2, batch size: 1 (with gradient accumulation for effective batch size 4)
• Learning rate: $\approx32$3 (constant), AdamW ($\approx32$4, $\approx32$5)
• Mixed-precision: fp16 for memory efficiency
• Max sequence length: 15,000 tokens (to capture full-length code and context)
• All samples are validated for PennyLane syntax correctness and minimal functional completeness (Basit et al., 25 Jul 2025).

4. Retrieval-Augmented and Graph-Based Generation Pipelines

In the precursor PennyLang work, context retrieval significantly enhances LLM code generation. The RAG approach involves:

• Corpus Embedding: All code+instruction+comment samples embedded via OpenAIEmbeddings (text-embedding-ada-002; 1536d).
• Dense Retrieval: Chroma DB, cosine similarity for top-$\approx32$6 samples per user query, with maximal marginal relevance ($\approx32$7) to promote context diversity.
• Prompt Assembly: Retrieved code samples are injected as explicit context into instruction prompts.

The GraphRAG extension (proposed) further constructs a bipartite graph with code samples and PennyLane API entities as nodes, connecting samples sharing API usage. A two-layer GNN (e.g., GAT) generates graph-aware retrieval embeddings, enabling subgraph-level contextualization for advanced prompt assembly.

Notably, while RAG boosts accuracy (e.g., GPT-4o Mini: 65.4 → 73.1 $\approx32$8, Qwen2.5-7B: 60.1 → 68.7), GraphRAG confers additional improvements by further reducing hallucinatory code and parameterization errors (Basit et al., 4 Mar 2025).

5. Evaluation Benchmarks and Comparative Performance

Evaluation methodology adapts standard functional correctness criteria with quantum-specific augmentations:

• Task Success Rate: pass@1 (snippet compiles and passes a minimal “oracle” functional test)
• Granular Scoring: Functionality (10-point), Syntax (10-point), Modularity (10-point). Composite score:

$\approx32$9

• Experimental Results (PennyCoder, held-out 264-task benchmark):

Model	Success	Failure	Accuracy (%)
PennyCoder (ours)	117	147	44.32
LLaMA 3.1-8B (base)	89	175	33.71
LLaMA 3.1-8B + RAG	106	158	40.15

• Latency (A100, batch=1, fp16): 0.18 s for PennyCoder (LoRA merged), 0.16 s for base LLaMA 8B. Quantized 8-bit: ∼1.2 s generation on CPU.
• Error Reduction: Hallucinations, wire index errors, and measurement mishandling are halved or quartered with RAG/GraphRAG (Basit et al., 4 Mar 2025, Basit et al., 25 Jul 2025).
• The pass@1 style metric is:

$W \in \mathbb{R}^{d\times d}$0

6. Deployment, Integration, and Engineering Guidance

PennyCoder is engineered for local or embedded environments, optimizing privacy and latency:

• Resource Profile: Full LLaMA 3.1-8B (fp16) is ∼16 GB; LoRA adapters 30 MB; 8-bit quantized model ∼8 GB.
• Integration: Deployed using HuggingFace PEFT: $W \in \mathbb{R}^{d\times d}$1
• Quantization: Post-training quantization to 4/8-bit (via bitsandbytes) supports CPU-centric and memory-constrained deployments.
• Prompting Schema: System prompts bundle user query, retrieved (or graph-augmented) code/comments, and explicit instructions for quantum task completion.
• Best Practices: Filter retrieval to relevant feature groups; leverage GraphRAG when possible for complex, multi-API workflows; extend the dataset with advanced templates and custom gate decompositions for higher task complexity (Basit et al., 4 Mar 2025).

7. Research Impact and Extensions

PennyCoder represents a shift toward efficient, privacy-preserving, domain-adapted LLMs for quantum programming. By centering on PennyLane, it addresses a gap unfilled by Qiskit-centric LLM research and sets a practical foundation for autonomous code synthesis in variational, QML, and QRL workflows. The system’s extensible design, including GraphRAG and multi-framework subgraphs, offers a research testbed for transfer across quantum programming languages.

A plausible implication is that PennyCoder’s paradigm—adapter-based model tuning on high-quality, annotation-rich corpora—can be generalized to other niche scientific domains where local operability and data privacy are paramount (Basit et al., 25 Jul 2025, Basit et al., 4 Mar 2025).