AnalogSeeker: LLM for Analog IC Design

• AnalogSeeker Foundation Model is an open-source Transformer-based model tailored for analog circuit design, utilizing a curated corpus and granular knowledge nodes.
• It employs a multi-agent reasoning framework to generate QTSA-formatted training examples, effectively addressing data scarcity and enabling structured technical reasoning.
• The model is fine-tuned using NSC-SFT to preserve knowledge while enhancing performance, achieving state-of-the-art results on AMSBench-TQA benchmarks.

The AnalogSeeker Foundation Model refers to a class of open-source Transformer-based LLMs specialized for analog circuit design, with the explicit objective of providing high-level design assistance and domain knowledge integration for analog IC tasks. Unlike conventional LLMs, which are typically trained on broad natural language corpora, AnalogSeeker leverages a targeted, curated corpus of analog circuit literature and a multi-stage, multi-agent knowledge distillation framework to address data scarcity and the structured, complex reasoning endemic to analog circuit engineering (Chen et al., 14 Aug 2025).

1. Domain-Specific Corpus Collection and Preprocessing

AnalogSeeker's corpus construction strategy is centered on exploiting the hierarchical knowledge structure of analog circuit theory. The process begins with the systematic selection of 20 canonical textbooks covering four stages: circuit theory, analog circuit fundamentals, analog integrated circuits, and advanced topics such as PLLs and ADCs. Nine foundational and eleven advanced volumes were processed using OCR (Mathpix API) to extract text, tables, and formulas while preserving the semantic outline (chapters, subchapters) and eliminating non-informative headers/footers.

The resulting corpus totals 7.26 million tokens and is structured into 2,698 "learning nodes" derived from two-level textbook headings, each averaging approximately 2,000 tokens. Coverage includes at least 12 classes of analog circuits (op-amps, filters, voltage regulators, and more). Each node contains compact, self-contained knowledge slices ideal for downstream distillation (Chen et al., 14 Aug 2025).

2. Granular Domain Knowledge Distillation via Multi-Agent Framework

To enable effective supervised fine-tuning (SFT) with high-quality, reasoning-rich labels, AnalogSeeker introduces a granular knowledge distillation paradigm. Each node in the corpus is transformed into multiple (five per node) QTSA-formatted examples:

• Q: Domain-relevant question
• T: Informal chain-of-thought reasoning (natural explanation)
• S: Explicit, structured solution steps
• A: Final answer

This transformation is orchestrated by a multi-agent system: a Q-agent generates questions targeting each node, an A-agent produces detailed answers with stepwise logical flow, and a P-agent refines, validates, and normalizes the output, filtering malformed or overly formula-reliant data. The final SFT dataset comprises 15,310 labeled examples totaling 112.65 million tokens. This approach yields compact, learnable units of domain knowledge conducive to LLM fine-tuning (Chen et al., 14 Aug 2025).

3. Model Architecture, Training Paradigm, and NSC-SFT Algorithm

AnalogSeeker is instantiated by fine-tuning a Qwen2.5-32B-Instruct model—a 32-billion parameter, RLHF-aligned, decoder-only Transformer—without architectural modification. To efficiently inject knowledge without catastrophic performance collapse or overfitting due to the relatively small SFT dataset, the training employs Neighborhood Self-Constrained Supervised Fine-Tuning (NSC-SFT). This algorithm augments the typical cross-entropy loss with a soft KL-divergence regularization that penalizes deviations of the fine-tuned model's output distribution from its reference checkpoint:

$$L(\theta) = L_\mathrm{CE}(\theta; x, y_\mathrm{label}) + \lambda \cdot D_\mathrm{KL}(p_\theta(\cdot|x) \| p_{\theta_0}(\cdot|x))$$

where $\lambda=0.1$, and $\theta_0$ are the reference (pre-SFT) parameters. Hardware-wise, training utilizes 8×NVIDIA H200 SXM GPUs (141 GB each) with DeepSpeed ZeRO-3 for efficient parameter distribution. The SFT dataset is packed to a maximum sequence length of 8,192 (Chen et al., 14 Aug 2025).

4. Experimental Results and Benchmarking

AnalogSeeker's efficacy is validated on AMSBench-TQA, a specialized multiple-choice benchmark for analog circuit knowledge evaluation. The key results:

Model	AMSBench-TQA Accuracy (%)
Qwen2.5-32B-Instruct	69.37
QwQ-32B (reasoning model)	81.54
DeepSeek-v3	84.41
GPT-4o	73.99
SFT only (Qwen2.5)	82.34
CPT+SFT (Qwen2.5)	82.74
AnalogSeeker (NSC-SFT)	85.04

AnalogSeeker achieves a +15.67 percentage point gain over the base model, and outperforms GPT-4o by +11.05 points, marking state-of-the-art pre-trained and commercial LLM performance for analog circuit tasks. Ablations confirm that SFT applied to a non-instruct (reasoning) base causes catastrophic performance degradation, while NSC-SFT robustly preserves and enhances task-specific knowledge (Chen et al., 14 Aug 2025).

In the operational amplifier design use case (Atelier framework), AnalogSeeker is able to iteratively suggest topology modifications and parameter tweaks to satisfy stringent gain, GBW, phase margin, and power requirements, acting in both topology- and sizing-agent roles. The model can autonomously propose compensation techniques and circuit adjustments, demonstrating capabilities beyond retrieval-augmented or heuristic systems (Chen et al., 14 Aug 2025).

5. Distinctive Technical Innovations

• Corpus Curation for Domain Scarcity: Canonical textbook mining and conversion to a granular, node-based structure directly address the lack of structured, labeled analog corpus data for LLMs.
• Multi-Agent Reasoning Distillation: Unstructured technical text is systematically transformed into a large set of supervised QTSA examples, encoding both solution structure and explanatory reasoning.
• Neighborhood Self-Constrained Fine-Tuning: By constraining output shifts via KL-divergence to a pre-trained reference, the method maximizes knowledge injection while minimizing catastrophic forgetting and mode collapse intrinsic to SFT scenarios with small, domain-heavy datasets.
• Open-Source Resource Provision: The preprocessed corpus, distilled SFT set, all training scripts, and the 32B checkpoint are public, enabling reproducibility, further fine-tuning, and meta-learning research (Chen et al., 14 Aug 2025).

6. Openness, Extensibility, and Usage

AnalogSeeker is distributed via HuggingFace, with all associated data and training recipes openly available. Researchers may download the NSC-SFT checkpoint, access tokenized datasets, and apply or adapt the NSC-SFT algorithm for extension to other analog (or even digital) circuit tasks, including downstream design automation for filters and mixed-signal blocks (Chen et al., 14 Aug 2025).

A plausible implication is that the combination of compositional, agent-driven distillation and reference-constrained tuning is transferable to other scientific or engineering domains with similar data characteristics. AnalogSeeker establishes a concrete, performant baseline for foundation models in analog circuit design, both for knowledge-intensive reasoning and active design synthesis responsibilities.

---

References:

• "AnalogSeeker: An Open-source Foundation LLM for Analog Circuit Design" (Chen et al., 14 Aug 2025)