ICGeo8K: Benchmark for IC Footprint Geometry

• ICGeo8K is a multimodal dataset that formalizes extracting structured IC footprint geometry from datasheet diagrams for automated PCB layouts.
• It comprises 8,608 samples, blending 4,138 real-world entries with 4,470 synthetic images to capture diverse package complexities.
• The benchmark’s three-phase task—pin counting, localization, and dimension estimation—demonstrates improved efficiency over manual EDA workflows.

Searching arXiv for ICGeo8K and related papers to ground the article. ICGeo8K is a multimodal benchmark and dataset for integrated-circuit footprint geometry understanding, introduced to support automated package geometry labeling directly from IC mechanical drawings. It formalizes a task in which a model receives a datasheet footprint diagram as an image and must recover the structured geometric description required to reconstruct the PCB land pattern. The benchmark was introduced alongside the framework "A LLM Powered Integrated Circuit Footprint Geometry Understanding" (Wang et al., 30 Jul 2025), which argues that existing large multimodal models perform poorly on dense engineering drawings because the target output is not a caption or class label but a pin-level geometric layout.

1. Definition and problem scope

ICGeo8K is a multimodal image–label dataset for IC footprint geometry understanding (Wang et al., 30 Jul 2025). In the formulation used by its authors, the input is a datasheet page image containing the footprint diagram, and the output is a structured representation of the package geometry. The benchmark targets direct interpretation of IC mechanical drawings rather than manual footprint construction in EDA tools.

The motivating application is PCB footprint generation. The paper states that accurate IC footprint geometry is essential for PCB layout, component placement, electrical connectivity, and manufacturability, and that incorrect pin geometry can cause impedance mismatch, parasitic effects, soldering defects, shorts, and assembly failures (Wang et al., 30 Jul 2025). It further identifies two core gaps: no dataset of paired datasheet footprint diagrams and structured pin-level geometry labels existed for large multimodal model training, and general-purpose large multimodal models are not spatially precise enough for engineering drawings with dense annotations, implicit layout rules, and misleading diagram elements.

Within that framing, ICGeo8K serves both as a supervised training resource and as a benchmark substrate. A plausible implication is that the dataset is intended to shift footprint interpretation from an artisanal EDA workflow toward a machine-readable geometric extraction workflow, but the paper’s concrete claims remain tied to datasheet-image input and structured geometric output.

2. Dataset composition and coverage

ICGeo8K contains 8,608 labeled samples in total, comprising 4,138 real-world samples manually curated from datasheets and 4,470 synthetic samples generated from EDA footprint files (Wang et al., 30 Jul 2025). The paper also refers to the synthetic subset as ICGeo8K-Syn. The real-world portion consists of footprint diagrams extracted from datasheet pages and corrected by PCB experts, while the synthetic portion is produced by transforming publicly available EDA footprint descriptions into clean, datasheet-style diagrams.

The dataset covers 10 IC package categories and spans pin counts from 1 to 800 pins (Wang et al., 30 Jul 2025). This wide complexity range is central to the benchmark’s design, because the target task must accommodate both sparse and dense pin layouts. The paper treats this range as evidence that the benchmark is not restricted to a narrow family of simple footprints.

The evaluation benchmark derived from the dataset is ICGeoQA, which contains 400 carefully curated real-world entries sampled from ICGeo8K and excluded from training (Wang et al., 30 Jul 2025). The authors further state that ICGeoQA reflects real-world package distribution, matches the package type distribution of a much larger pool of 200,000 Digi-Key IC entries, and shares the same pin-count distribution as ICGeo8K. This distributional alignment is used to support the benchmark’s realism.

Component	Size	Description
ICGeo8K total	8,608	Multimodal labeled samples
Real-world subset	4,138	Manually curated from datasheets
Synthetic subset	4,470	Generated from EDA footprint files
ICGeoQA	400	Curated real-world evaluation entries

The distinction between real and synthetic data is methodologically important. The paper states that synthetic diagrams are cleaner and more uniform, whereas real datasheets contain multiple diagrams, inconsistent label placement, varying fonts, and unrelated technical content (Wang et al., 30 Jul 2025). This suggests that ICGeo8K is designed to separate basic geometric learning from robustness to document noise.

3. Task formulation and annotation schema

The benchmark defines IC footprint geometry understanding as a three-part structured extraction task (Wang et al., 30 Jul 2025). Given a datasheet diagram, the model must infer the number of pins, the center coordinates of each pin, and the dimensions of each individual pin. The paper presents these as progressive sub-questions in a chain-of-thought-style conversation: first count the pins, then locate each pin center relative to a reference origin, and then estimate each pin’s width and length in millimeters.

This decomposition is intended to mimic the reasoning process used by human engineers when converting a datasheet figure into an EDA footprint (Wang et al., 30 Jul 2025). The paper emphasizes that the sequencing matters because errors in the counting stage propagate to localization and dimension estimation.

The annotations are pin-level geometric labels derived from EDA source files and corrected against datasheet diagrams by experts. The label structure contains pin count, per-pin center coordinates, and per-pin dimensions (Wang et al., 30 Jul 2025). The EDA files are typically in XML and contain location and size information for pins. In the training format described in the appendix, each supervised example consists of an image paired with a JSON-format QA conversation aligned to the corrected EDA labels.

This annotation design gives ICGeo8K a hybrid character: it is both an image understanding dataset and a structured geometric extraction benchmark. A plausible implication is that the dataset is closer to engineering diagram parsing than to conventional visual question answering, even though the supervision is expressed through conversational formatting.

4. Data collection and synthetic generation

The paper describes a four-step pipeline for constructing the real-world portion of ICGeo8K (Wang et al., 30 Jul 2025). IC models are compiled from Digi-Key, datasheets are downloaded using part numbers, and matching EDA footprint files are retrieved from Ultra Librarian. Because the footprint diagram page is often buried within a datasheet, Gemini 2.0 is used to suggest candidate pages, after which engineers manually verify the page selection.

Once the relevant page is identified, the verified datasheet pages are extracted as images, and the EDA files are parsed to obtain pin count, pin centers, and pin dimensions (Wang et al., 30 Jul 2025). PCB engineers then correct mismatches between EDA-derived labels and datasheet diagrams. The paper notes that these engineers were paid $0.61 per processed sample. After filtering and correction, the retained corpus contains 4,138 real-world entries spanning 10 package types.

The synthetic subset is generated because real-world annotation is costly (Wang et al., 30 Jul 2025). The generator converts EDA descriptions into clean, uniform, datasheet-style images with aligned annotations. According to the paper, this synthetic data is used to teach basic geometric reasoning before exposing the model to noisy real datasheets.

The real-world and synthetic subsets therefore correspond to distinct roles in the benchmark ecosystem. Real-world entries provide practical difficulty and document variability; synthetic entries provide controlled geometric supervision. This suggests a deliberate separation between annotation fidelity and visual realism, though the paper’s explicit claim is that the synthetic-first strategy improves training.

5. Evaluation protocol and reported performance

ICGeo8K is evaluated through both sub-task metrics and an overall layout metric (Wang et al., 30 Jul 2025). For overall reconstruction quality, the paper defines a custom layout metric, $IoU_{IC}$, intended as</p> <p>$IoU_{IC} = \frac{Area_{pred} \cap Area_{label}}{Area_{pred} \cup Area_{label}}.$</p> <p>Here,$Area_{pred}$$is the reconstructed area from predicted pin geometry, and$$Area_{label}$$is the reconstructed area from ground-truth pin geometry. A higher$$IoU_{IC}$$indicates closer agreement between predicted and true footprint layout.</p> <p>The three sub-tasks are evaluated separately. Pin counting uses <a href="https://www.emergentmind.com/topics/meta-autoencoder-mae" title="" rel="nofollow" data-turbo="false" class="assistant-link" x-data x-tooltip.raw="">MAE</a> and <a href="https://www.emergentmind.com/topics/root-mean-squared-error-rmse" title="" rel="nofollow" data-turbo="false" class="assistant-link" x-data x-tooltip.raw="">RMSE</a>; pin center localization uses the average Euclidean distance between predicted and true pin centers, denoted$$d_{pin}$$; and pin dimension estimation uses average per-pin <a href="https://www.emergentmind.com/topics/semantic-intersection-over-union-iou" title="" rel="nofollow" data-turbo="false" class="assistant-link" x-data x-tooltip.raw="">IoU</a>, denoted$$IoU_{pin}$$(<a href="/papers/2508.03725" title="" rel="nofollow" data-turbo="false" class="assistant-link" x-data x-tooltip.raw="">Wang et al., 30 Jul 2025</a>).</p> <p>The benchmark paper reports poor performance from several general-purpose large multimodal models on ICGeoQA. On the overall metric$$IoU_{IC}$$, <a href="https://www.emergentmind.com/topics/vocabulary-assistant-llm-gpt-4o" title="" rel="nofollow" data-turbo="false" class="assistant-link" x-data x-tooltip.raw="">GPT-4o</a> achieves 11.1$$d_{pin} = 1.11$for pin centers, and$IoU_{pin} = 88.0\%$$for dimensions.</p> <div class='overflow-x-auto max-w-full my-4'><table class='table border-collapse w-full' style='table-layout: fixed'><thead><tr> <th>Model / Metric</th> <th style="text-align: right">Reported result</th> </tr> </thead><tbody><tr> <td>GPT-4o$$IoU_{IC} = \frac{Area_{pred} \cap Area_{label}}{Area_{pred} \cup Area_{label}}.$0 11.1% Gemini 2.0 $IoU_{IC} = \frac{Area_{pred} \cap Area_{label}}{Area_{pred} \cup Area_{label}}.$1 4.5% DeepSeek-VL2 $IoU_{IC} = \frac{Area_{pred} \cap Area_{label}}{Area_{pred} \cup Area_{label}}.$2 1.5% Qwen2-VL-7B $IoU_{IC} = \frac{Area_{pred} \cap Area_{label}}{Area_{pred} \cup Area_{label}}.$3 1.7% LLM4-IC8K $IoU_{IC} = \frac{Area_{pred} \cap Area_{label}}{Area_{pred} \cup Area_{label}}.$4 71.6%

The paper presents these results as evidence that general-purpose large multimodal models lack the domain-specific geometric grounding required for engineering drawing interpretation (Wang et al., 30 Jul 2025). It also states that all baseline general models remain below 20% overall IoU. A plausible implication is that ICGeo8K exposes a failure mode that is obscured in broader multimodal benchmarks: high-level visual reasoning does not imply reliable geometric extraction from technical diagrams.

6. LLM4-IC8K and the benchmark’s methodological role

ICGeo8K is tightly coupled to the two-stage training framework LLM4-IC8K, built on Qwen2-VL-7B (Wang et al., 30 Jul 2025). In the first stage, the model is trained on the synthetic subset to learn fundamental geometric reasoning from relatively clean and uniform annotation patterns. In the second stage, it is fine-tuned on real datasheet samples, which are harder because they involve varying resolution, multiple visual styles, different annotation conventions, clutter, irrelevant content, and multiple diagrams on a page.

The training prompts are organized as a three-round CoT-like dialogue corresponding to the three sub-tasks: pin count, pin center coordinates, and pin dimensions (Wang et al., 30 Jul 2025). The paper argues that this sequence mirrors human reasoning and enables progressive inference of the footprint geometry.

Ablation results reported in the paper indicate that both the dialogue structure and dataset order are consequential. The best strategy is the chained conversation $$IoU_{IC} = \frac{Area_{pred} \cap Area_{label}}{Area_{pred} \cup Area_{label}}.$$5, which yields the best overall $$IoU_{IC} = \frac{Area_{pred} \cap Area_{label}}{Area_{pred} \cup Area_{label}}.$$6 of 71.6%, while splitting the tasks into separate rounds causes substantial performance degradation (Wang et al., 30 Jul 2025). The best training order is synthetic first and real-world second. The paper states that this two-stage strategy improves overall $$IoU_{IC} = \frac{Area_{pred} \cap Area_{label}}{Area_{pred} \cup Area_{label}}.$$7 by 10%, 5%, and 189.9% compared with certain alternative training orders in the ablation table.

In this sense, ICGeo8K is not merely a static benchmark. It is also the empirical basis for a claim about how geometric reasoning should be scaffolded in multimodal models for engineering tasks. The paper’s position is that domain-specific supervision and structured reasoning are necessary conditions for footprint geometry understanding (Wang et al., 30 Jul 2025).

7. Significance, limitations, and relation to engineering workflows

The significance of ICGeo8K lies in its focus on automated package geometry labeling directly from IC mechanical drawings, which the paper characterizes as previously unsupported by existing methods or datasets (Wang et al., 30 Jul 2025). The benchmark targets a concrete PCB-engineering problem rather than a generic multimodal reasoning task, and its label space is explicitly machine-readable and pin-level.

The paper also situates ICGeo8K relative to manual EDA workflows. It reports approximate runtime and accuracy comparisons for manual footprint creation in Altium, Autodesk EAGLE, and KiCad, against LLM4-IC8K (Wang et al., 30 Jul 2025). Altium is listed at about 7 minutes per sample with $$IoU_{IC} = \frac{Area_{pred} \cap Area_{label}}{Area_{pred} \cup Area_{label}}.$$8, Autodesk EAGLE at about 15 minutes per sample with $$IoU_{IC} = \frac{Area_{pred} \cap Area_{label}}{Area_{pred} \cup Area_{label}}.$$9, KiCad at about 15 minutes per sample with $Area_{pred}$0, and LLM4-IC8K at 0.26 minutes per sample with $Area_{pred}$1. The authors state that this corresponds to up to a 58× reduction in time relative to traditional EDA workflows.

At the same time, the benchmark is framed around limitations of current large multimodal models rather than around fully solved automation. The paper identifies several weaknesses in general-purpose models: poor fine-grained spatial reasoning, inability to interpret dense engineering annotations, confusion between pin spacing and pin size, failure to correctly count pins when omission symbols are used, difficulty recognizing numeric values precisely, and lack of domain-specific priors for footprint geometry (Wang et al., 30 Jul 2025). These observations function as both benchmark motivation and error taxonomy.

ICGeo8K therefore occupies a specific position within multimodal benchmarking. It is neither a generic document-understanding corpus nor a classical CAD dataset. Instead, it is a task-specific benchmark for recovering exact geometric structure from technical drawings. This suggests that its long-term relevance may depend not only on raw benchmark scores but also on whether future systems can close the remaining accuracy gap between automated inference and high-accuracy manual EDA tools.