AttentionCap: Transformer Based Capacitance Matrix Learning Toward Full-Chip Extraction

Abstract: As capacitance extraction accuracy of rule-based pattern matching becomes difficult to sustain at advanced nodes, a growing trend emerges to develop deep-learning-based 2D capacitance models. However, existing MLP- and CNN-based methods constrain their input to fixed metal-layer combinations in a specific process node, limiting their usability in practice. Recognizing the inherent similarity between capacitance matrix and the prevailing attention mechanism, we propose AttentionCap, a customized Transformer for capacitance matrix learning, with a Gram representation framework, a physics-aligned symmetric-attention output layer, and a novel normalized Laplacian loss. We also introduce a process-node embedding to enable multi-node learning. Trained on synthetic data, AttentionCap attains 0.67\%/3.99\% self/coupling-capacitance error on unseen real designs under a multi-layer and multi-node setting, surpassing the CNN-Cap baseline with 4.6$\times$/5.7$\times$ lower self/coupling error and 192$\times$ faster inference speed. A pretrained AttentionCap accurately transfers to an unseen node with only 5K samples and 4K finetuning steps. With sufficient accuracy on unseen real designs and strong transferability to new process nodes, AttentionCap offers highly practical value for modern EDA workflows. Code and data are available at https://github.com/THU-numbda/AttentionCap.

• The paper introduces AttentionCap, a Transformer architecture that learns full-chip capacitance matrices in a single inference pass.
• The paper demonstrates superior accuracy (self error: 0.67%, coupling error: 3.99%) and 192× faster inference compared to CNN methods.
• The paper achieves robust cross-node transferability by using process-node embeddings and a normalized Laplacian loss for adaptation across technology nodes.

Transformer-Based Capacitance Matrix Learning for Full-Chip Parasitic Extraction

Motivation and Limitations of Prior Approaches

The need for accurate and scalable parasitic capacitance extraction is central to modern VLSI design, directly affecting signoff, timing, signal integrity, and power analysis. Traditional rule-based pattern-matching approaches and field solvers such as BEM and FRW offer accuracy, yet fail to scale efficiently with increasing design complexity and process node miniaturization. Recent deep-learning-based methods, notably those employing MLPs or CNNs on density grid representations (e.g., CNN-Cap [yang2021cnn]), improved automation but suffer from intrinsic constraints:

• Input/Output Rigidity: Fixed input dimensions necessitate separate models for each conductor and metal-layer combination.
• Combinatorial Model Proliferation: Process coverage across all metal layers drives up the number of necessary models, impacting practical workflow.
• Limited Expressivity: Modeling only individual self or coupling capacitances, these approaches neglect the holistic $n \times n$ mutual interdependence inherent in Maxwell’s equations.
• Generalization Deficits: Generalization across process nodes is not addressed, hampering transferability and increasing the cost for new node bring-up.

The AttentionCap Architecture: Physics-Aligned Transformer for Capacitance Modeling

AttentionCap reframes the capacitance matrix learning problem as an $n$-to-$n$ mapping, exploiting the structural analogy between the interaction matrix in Transformers’ self-attention mechanism and the physical capacitance matrix. The architecture incorporates several technical components:

• Order Equivariance: Omitting positional encodings ensures permutation equivariance, matching underlying physics.
• Gram Representation: Capacitance matrices, being symmetric and positive semidefinite, admit a dot-product (Gramian) decomposition. The model learns "capacitance embeddings" for each conductor; their (scaled) dot products form the predicted matrix.
• Symmetric-Attention Output: The final output layer computes the entire matrix in a single inference pass, enforcing symmetry via dot-product and explicit Laplacian properties.
• Process-Node Embedding: A learnable embedding vector encodes process-node-specific information, enabling joint training across multiple technology nodes and fast transfer/adaptation via light retraining.
• Normalized Laplacian Loss: A novel loss function normalizes errors with respect to the magnitude of ground-truth elements, improving stability and accuracy given wide dynamic range.

  Figure 1: Modern Transformer architecture.

  

  Figure 2: Overview of the AttentionCap framework for capacitance matrix learning.

Empirical Evaluation: Benchmarks and Generalization

On synthetic and real-design datasets spanning 7nm to 65nm nodes, AttentionCap exhibits several notable empirical properties:

• Superior Accuracy: On unseen real-design test sets in the multi-layer, multi-node regime, the model achieves mean self/coupling-capacitance errors of 0.67%/3.99%, with 4.6$\times$/5.7$\times$ lower error than CNN-Cap.
• One-Pass Full Matrix Prediction: Unlike CNN-Cap, which requires $n^2$ passes per sample, AttentionCap outputs the full matrix in a single forward inference.
• Massive Efficiency Gains: The method is 192$\times$ faster at inference compared to density-grid CNNs.
• Variable-Length, All-Layer, All-Node Support: The architecture natively accommodates any number of conductors and arbitrary metal-layer/process-node combinations.
• Strong Transferability: Multi-node pretraining produces a process-agnostic backbone. With 5K samples and 4K finetuning steps on a new node, transfer accuracy surpasses models trained from scratch with up to 240K steps.

  Figure 3: AttentionCap learns meaningful Gram representations for each conductor (visualized via 3D PCA on a unit sphere, line thickness reflecting their dot products); its internal attention patterns match the target capacitance matrix.

  

  Figure 4: Example cross-sections from ASAP7 real-design test set, in local 3D view (left) and local top view (right).

  

  Figure 5: Strong transferability of AttentionCap: with only 5K samples and 4K finetuning steps for new process node (7nm), the pretrained model (on 15nm+28nm+65nm) outperforms the best model trained from scratch.

Theoretical and Practical Implications

The approach demonstrates that self-attention mechanisms in Transformers generalize beyond the domains of language and vision, capturing the physics of $n$-body interactions even in settings (capacitance extraction) where spatial locality and physical symmetry are vital. The order-invariant sequence model removes the need for manual pattern design and combinatorial model proliferation; a single model can be maintained for all metal-layer patterns in a process node, extended naturally to new nodes via lightweight adaptation.

The Gram representation learning provides not only functional accuracy but interpretable embeddings, aligning the learned model structure with physical intuition—conductors with higher coupling have more similar embeddings.

The process-node embedding strategy generalizes to other technology/document/process adaptation tasks, suggesting a scalable path to device- and layout-agnostic EDA tools.

Future Outlook

AttentionCap's paradigm opens several research directions:

• Non-Manhattan/Complex Geometry Support: Incorporation of higher-dimensional or graph-based embeddings to handle arbitrary 2D/3D and nonrectilinear structures.
• Integration with 3D Extraction and GNNs: Hybridization with graph neural networks for vertical coupling modeling.
• Data-Efficient Pretraining: Systematic study of synthetic-to-real transfer, including active learning and uncertainty quantification for solver data generation.
• Wider EDA Application: Application to resistance/inductance matrix extraction, differential and process-variation-aware prediction, and co-simulation with circuit-level models.

Conclusion

AttentionCap provides a refined, physics-aligned Transformer architecture for full-chip capacitance matrix extraction, overcoming the key limitations of prior deep-learning methods in EDA. Its empirical results—0.67%/3.99% self/coupling error, 192$\times$ faster inference, and robust inter-node transfer—demonstrate its viability as a unified, scalable solution for practical capacitance extraction workflows. The alignment of deep learning architectures with underlying physics, as executed in this work, sets a solid precedent for subsequent advances in AI-driven EDA automation.

(2606.08161)