MuaLLM: Circuit Design Multimodal Agent

• MuaLLM is an open-source multimodal agent designed for circuit design assistance, integrating hybrid retrieval, adaptive indexing, and iterative ReAct workflows.
• It processes circuit-design PDFs to extract textual and visual evidence, using LLM-generated image descriptions to enhance multimodal retrieval.
• The system demonstrates high recall and efficiency by decoupling inference from corpus size, achieving up to 90.1% recall and faster response times compared to full-context prompting.

Searching arXiv for MuaLLM and closely related papers to ground the article. MuaLLM is an open-source multimodal LLM agent for circuit design assistance that integrates a hybrid retrieval-augmented generation framework with an adaptive vector database of circuit design research papers. It is designed as a question-answering design assistant that interprets complex circuit queries, grounds its responses in retrieved literature, processes both textual and visual evidence, and uses a Reason + Act workflow for iterative reasoning, goal-setting, and multi-step information retrieval. The system is evaluated on two custom datasets, RAG-250 and Reasoning-100, and the paper reports 90.1% recall on RAG-250 and 86.8% accuracy on Reasoning-100 (Abbineni et al., 11 Aug 2025).

1. Definition and problem setting

MuaLLM is situated in the technical workflow of circuit design, where literature review, architecture comparison, and design tradeoff analysis depend on evidence distributed across prose, equations, tables, plots, and circuit schematics. The paper frames this as a domain in which conventional LLM use is inadequate because generic models are vulnerable to hallucination, direct full-context prompting is constrained by fixed context windows, and many circuit-design questions require iterative retrieval and reasoning rather than single-shot answer generation (Abbineni et al., 11 Aug 2025).

The system is therefore designed to solve a combined problem: scalable, literature-grounded, multimodal, multi-step circuit-design assistance. Its central claim is that circuit-design support requires more than a plain chatbot. It requires reliable retrieval over large technical corpora, tool use, and multimodal access to visual evidence such as graphs, equations, and schematics. In the paper’s formulation, MuaLLM addresses this through three coupled layers: a multimodal document database and preprocessing pipeline, a hybrid contextual RAG stack, and a ReAct-based agentic controller (Abbineni et al., 11 Aug 2025).

This positioning is important because MuaLLM is not introduced as a new foundation model trained from scratch. It is a systems-level architecture that decouples retrieval from inference, allowing reasoning over arbitrarily large corpora while avoiding the cost and latency growth associated with direct prompt stuffing. A stated practical consequence is that, at the maximum context length supported by standard LLMs, MuaLLM remains up to 10x less costly and 1.6x faster while maintaining the same accuracy (Abbineni et al., 11 Aug 2025).

2. Document processing, indexing, and hybrid contextual RAG

The document pipeline begins with circuit-design PDFs. Using the unstructured library, MuaLLM extracts both text and images from PDF documents. The text is cleaned and split into overlapping paragraph-level chunks; the paper states that paragraph-level chunking yielded better search results than character-level chunking, and overlap is used to preserve semantic continuity. Images are retained because circuit papers often encode critical information in schematics, plots, and specification tables (Abbineni et al., 11 Aug 2025).

For visual indexing, MuaLLM does not rely on generic image embeddings alone. Instead, an LLM generates detailed image descriptions, including image type and domain-specific technical details. Those descriptions are embedded using Voyage-2 and stored with metadata, including the image path, in a unified vector database. The paper notes that CLIP was initially tried but was limited for domain-specific circuit content. The generated descriptions are used to enhance retrieval recall and indexing of visual content rather than as final factual outputs (Abbineni et al., 11 Aug 2025).

The retrieval layer is termed “hybrid contextual retrieval-augmented generation.” “Hybrid” refers to the parallel use of semantic search and BM25 keyword search. “Contextual” refers to preserving or restoring broader context for retrieved chunks and to the use of a contextual caching mechanism inspired by Anthropic’s contextual retrieval. At query time, semantic search supports concept-level matching, while BM25 supports exact technical terminology. The merged result set is then reranked with a Cohere reranker before being passed to a generative model such as GPT-4o; the paper also notes that Claude 3.5 Sonnet can be swapped in because the generation stage is modular (Abbineni et al., 11 Aug 2025).

The paper is systems-oriented rather than mathematically formal. It states that retrieved candidates are assigned weighted scores based on retrieval source, and that semantic retrieval may be weighted more heavily when conceptual relevance matters, but it does not provide an explicit retrieval-scoring formula, an objective function, or a learned policy equation for this stage. A plausible implication is that MuaLLM’s contribution lies more in orchestration and retrieval design than in a new formal retrieval model (Abbineni et al., 11 Aug 2025).

3. ReAct agent architecture and adaptive tool use

MuaLLM’s defining feature is its ReAct-based agentic workflow. Rather than operating as a static RAG pipeline, it executes an iterative thought–action–observation loop. The agent reasons about what evidence is missing, invokes tools, inspects retrieved results, and may continue retrieval or database expansion before producing a final response. This is presented as a response to the fact that circuit-design questions often require goal decomposition, citation chasing, and multi-step evidence aggregation (Abbineni et al., 11 Aug 2025).

The paper describes four main tools. search_db performs hybrid retrieval over the local corpus and supports both text and visual search. paper_fetcher downloads missing papers from public repositories such as Google Scholar and arXiv. search_db --load_data preprocesses newly fetched PDFs and updates the vector database in real time. Netlist Generator converts schematic images into SPICE-compatible netlists. Together, these tools make the database adaptive rather than static, allowing the system to update its literature base without human intervention (Abbineni et al., 11 Aug 2025).

The adaptive loop is illustrated with examples. In one case, the agent determines that a required comparison depends on a paper not present in the current database, fetches that paper, processes it, and only then completes the answer. In another, to determine which bandgap reference circuit has the lowest power, it first retrieves candidate titles and then iteratively inspects the sources for the specific power values rather than answering from partial information. These examples are used to distinguish the full ReAct agent from plain multimodal RAG (Abbineni et al., 11 Aug 2025).

This design has two implications. First, MuaLLM is explicitly an agent rather than a retrieval-backed chatbot. Second, its reasoning quality is tightly coupled to tool availability and retrieval quality. The paper’s qualitative comparison between MRAG and the ReAct agent supports this reading: MRAG can retrieve useful evidence, but the ReAct agent is better at recognizing missing steps and initiating additional retrieval (Abbineni et al., 11 Aug 2025).

4. Multimodality, schematic understanding, and netlist generation

MuaLLM is multimodal in a retrieval-centric sense. It ingests text, equations, tables, plots, and circuit schematics from PDF literature and stores them in a shared retrieval space. The paper evaluates image, equation, and text citation performance separately, indicating that multimodality is not treated as a secondary feature but as a core requirement of circuit-design assistance (Abbineni et al., 11 Aug 2025).

A distinctive subsystem is the Netlist Generator. Its purpose is to convert circuit schematic images into SPICE-compatible netlists. The pipeline comprises a YOLO-based detector that identifies components, labels, and bounding boxes; OpenCV Connected Components with Stats to identify connected wire clusters after component removal; validation of candidate nodes by intersection with the edges of two or more components; assignment of connected nodes to each component; clustering of ground-connected nodes; and generation of the final structured netlist (Abbineni et al., 11 Aug 2025).

The paper presents this tool as more than a convenience feature. It is tied to the claim that MuaLLM can support rapid, no-human-in-the-loop database generation, thereby addressing a bottleneck in circuit AI workflows where labeled data often requires manual netlist construction or simulation-based dataset generation. This suggests that MuaLLM is intended to operate not only as a literature QA agent but also as an upstream data-construction assistant for circuit-design research (Abbineni et al., 11 Aug 2025).

The multimodal strategy also reveals a practical engineering judgment. Instead of depending on a domain-generic vision embedding pipeline, MuaLLM uses LLM-generated descriptions of extracted images to improve search recall for circuit-specific visual content. The paper acknowledges that such descriptions may hallucinate, but confines them to indexing and retrieval support rather than direct answer generation. A plausible implication is that the authors prioritized retrieval effectiveness over strict visual grounding at index time (Abbineni et al., 11 Aug 2025).

5. Benchmarks, evaluation protocol, and reported results

Because the paper identifies no suitable public benchmark for this problem, it introduces two custom datasets curated with domain experts. RAG-250 contains 250 question-answer pairs derived from Bandgap Reference circuit literature, with expert-verified answers. It is used to evaluate retrieval-backed response quality and citation performance. Reasoning-100, also called Reas-100, contains 100 reasoning questions designed to test multi-step analytical capability in circuit design (Abbineni et al., 11 Aug 2025).

The RAG-250 evaluation is broken down into 250 overall-response queries, 130 image-citation queries, 45 equation-citation queries, and 75 text-citation queries. The reported metrics are Overall Response (Recall), Image Citation (Precision), Equation Citation (Recall), and Text Citation (F1 Score). Reas-100 is evaluated by manually comparing system outputs with expert-generated solutions, and the reported metric is accuracy (Abbineni et al., 11 Aug 2025).

Benchmark	Purpose	Reported result
RAG-250	Retrieval, synthesis, and citation in BGR literature	90.1% recall with GPT
Reas-100	Multi-step reasoning in circuit design	86.8% accuracy with GPT
Netlist Generator	Schematic-to-netlist conversion	Precision 0.94, Recall 0.91, F1 0.92, weighted mAP 0.99

On RAG-250, the multimodal RAG configuration achieves 88.1% recall with Claude and 90.1% recall with GPT. Image citation precision is reported as 89.8% with Claude and 93.02% with GPT; equation citation recall is 79.0% with Claude and 73.67% with GPT; text citation F1 is 93.0% with Claude and 96.0% with GPT. On Reas-100, MuaLLM achieves 84.0% accuracy with Claude and 86.8% with GPT. The main baseline is direct full-context prompting with GPT-4o or Claude Sonnet 3.5, which the paper treats as increasingly impractical as corpus size grows because of context-window limits and cost scaling (Abbineni et al., 11 Aug 2025).

The efficiency results are central to the paper’s argument. Figure 1 is summarized as showing that the full-context baseline becomes increasingly expensive and slow with more pages per query, eventually reaching the context-window limit, whereas MuaLLM retrieves only relevant chunks. The paper also reports a one-time preprocessing cost of about $0.21 per paper. This supports the paper’s broader claim that MuaLLM decouples query-time inference from total corpus size (Abbineni et al., 11 Aug 2025).

The evaluation section is strong on benchmark creation and system-level metrics, but it is limited in another respect: the paper does not provide formal ablation studies isolating the contributions of multimodality, hybrid retrieval, reranking, ReAct, or adaptive database updates. This suggests that the strongest empirical evidence concerns end-to-end utility rather than component-level causal attribution (Abbineni et al., 11 Aug 2025).

6. Naming, position in the literature, and limitations

The name MuaLLM refers specifically to the circuit-design agent described in “MuaLLM: A Multimodal LLM Agent for Circuit Design Assistance with Hybrid Contextual Retrieval-Augmented Generation” (Abbineni et al., 11 Aug 2025). It should not be conflated with “MaLLaM,” a Malaysia-centered LLM family trained from scratch for Malaysian context (Zolkepli et al., 2024), “MuMu-LLaMA,” a multimodal music understanding and generation system built around LLaMA (Liu et al., 2024), or “MALLM,” a multi-agent LLMs framework for multi-agent debate (Becker et al., 15 Sep 2025).

Within the literature represented here, MuaLLM is distinctive because it is not primarily a foundation-model pretraining effort, a multimodal generation model, or a debate framework. It is a domain-specific, literature-grounded agent whose core novelty lies in combining multimodal indexing, hybrid retrieval, adaptive corpus expansion, and ReAct-style tool use for circuit-design assistance (Abbineni et al., 11 Aug 2025).

The paper also has clear limitations. It does not specify the vector database backend, exact embedding dimensions, exact prompt templates, hardware configuration, or a mathematical formulation of the hybrid retrieval weighting. It explicitly lacks controlled ablation studies. Equation citation performance is weaker than image and text citation. Some comparisons to adjacent systems are unavailable because relevant baselines are not open-sourced or solve different tasks. These omissions make MuaLLM more useful as a systems blueprint and benchmark contribution than as a fully specified algorithmic reference (Abbineni et al., 11 Aug 2025).

Its significance nevertheless follows from a narrow but consequential claim: circuit-design assistance benefits from an agent that can search, cite, compare, follow references, interpret visual content, and update its own literature database. In that sense, MuaLLM marks a shift from passive LLM-based technical QA toward tool-using, retrieval-grounded design support specialized for circuit research (Abbineni et al., 11 Aug 2025).