Physics-Structured Linear Attention (PSLA)

• PSLA is a physics-aware linear-attention mechanism that integrates Manhattan-distance decay into transformer kernels to enforce a physically motivated inductive bias.
• It reparameterizes standard linear attention with separable, position-dependent decay factors, preserving linear-time complexity while handling sparse EDA data.
• Empirical results demonstrate that PSLA improves data efficiency, stability under distribution shifts, and offers significant memory and runtime gains in large-scale chip design.

Physics-Structured Linear Attention (PSLA) is a physics-aware linear-attention mechanism introduced as the architectural component of PhysEDA for electronic design automation (EDA) tasks (Yang, 11 May 2026). It incorporates a physically motivated spatial prior directly into a linear-attention transformer by exploiting the observation that pairwise electrical and routing interactions decay approximately exponentially with Manhattan distance on chip layouts. In this formulation, the attention kernel is reweighted by separable position-dependent factors, so that the model prefers nearby, physically plausible interactions while retaining linear-time scaling. Within the broader literature on efficient attention for scientific and engineering domains, PSLA exemplifies an approach in which latent structure induced by physical priors is expressed through linear-attention algebra; a related reinterpretation appears in work showing that Transolver’s Physics-Attention can be recast as a special case of linear attention (Hu et al., 9 Nov 2025).

1. Conceptual basis and motivation

PSLA was introduced to address two bottlenecks identified in learning-based EDA systems (Yang, 11 May 2026). First, vanilla attention has quadratic complexity in sequence length, which limits scaling on large grids or long placement sequences. Second, EDA data are sparse and structured by physics, so models trained only from data may overfit statistical noise and amplify weak long-range correlations rather than recover the correct spatial inductive bias.

The central physical prior used by PSLA is that pairwise electrical and routing interactions decay approximately exponentially with Manhattan distance. The paper motivates this most directly through power delivery network behavior and decoupling-capacitor placement, where transfer impedance decays with path length through the mesh. Because orthogonal VLSI routing makes path length well approximated by Manhattan distance, and because horizontal and vertical routing layers can be asymmetric, the prior takes direction-dependent decay rates in the horizontal and vertical axes.

PhysEDA treats PSLA and Potential-Based Reward Shaping (PBRS) as complementary components with distinct roles. PSLA provides an architecture-level inductive bias for representation learning, whereas PBRS is a training-level reward-shaping method for reinforcement learning. In that decomposition, PSLA is the component that directly targets scale, memory, and data efficiency (Yang, 11 May 2026).

A common misconception is to regard PSLA as a wholly new attention family. The formulation in PhysEDA explicitly states otherwise: PSLA does not introduce a new attention algorithmic family; it is a physics-structured reparameterization of linear attention. The significance of that claim is methodological rather than terminological: the physical prior is inserted inside the linear-attention kernel rather than imposed only through positional encodings or post hoc regularization.

2. Mathematical formulation

The physical hypothesis underpinning PSLA is expressed through a Manhattan-distance decay law. For a point $i$ receiving a response from source $j$ at frequency $f$,

$|Z_{\mathrm{tr}(i,j,f)| \approx Z_0(f)\cdot \exp\!\bigl(-\alpha_x|x_i-x_j| - \alpha_y|y_i-y_j|\bigr)$

and the appendix derives the same form from a transmission-line mesh PDN model via

$|Z_{\mathrm{tr}(i,j,f)| \propto \exp\!\bigl(-\kappa(f)\,n\bigr), \qquad n = |x_i - x_j| + |y_i - y_j|.$

PSLA starts from standard linear attention with a nonnegative feature map $\phi$:

$$\mathrm{LinAttn}(Q,K,V)=\frac{\phi(Q)[\phi(K)^\top V]}{\phi(Q)\phi(K)^\top\mathbf{1}}$$

and modifies it as

$$\mathrm{PSLA}(Q,K,V)=\frac{(\phi(Q)\odot D_Q)[(\phi(K)\odot D_K)^\top V]}{(\phi(Q)\odot D_Q)(\phi(K)\odot D_K)^\top\mathbf{1}}$$

with

$$D_Q[i]=\exp(-\alpha_x x_i-\alpha_y y_i), \qquad D_K[j]=\exp(+\alpha_x x_j+\alpha_y y_j),$$

where $(x_i,y_i)\in[0,1]^2$ are normalized chip coordinates and $j$0 denotes elementwise multiplication (Yang, 11 May 2026).

The effective pairwise weight is therefore proportional to

$j$1

This yields a factorization into a content term and a physically structured spatial decay term. Because the decay separates into $j$2 and $j$3, it can be absorbed into queries and keys separately, avoiding an explicit $j$4 distance matrix. That separability is the core algebraic property enabling PSLA to preserve linear-attention scaling while encoding a Manhattan-distance prior.

The main implementation uses a rank-1 directional approximation rather than the exact symmetric kernel. Specifically, the realized term is

$j$5

whereas the physically symmetric kernel would be

$j$6

The appendix describes an exact symmetric reconstruction using bidirectional prefix sums, but the main experiments use the rank-1 form because it is much faster and empirically effective (Yang, 11 May 2026).

3. Mechanism and implementation

The modification from standard linear attention to PSLA is operationally simple. One begins with linear attention using a positive feature map $j$7, augments each token with chip coordinates $j$8, computes the decay gates $j$9 and $f$0, replaces $f$1 and $f$2 by their elementwise products with those gates, performs the usual linear-attention contraction, and normalizes with the corresponding scalar denominator (Yang, 11 May 2026).

Several implementation choices are specified. PSLA uses $f$3 with $f$4; LayerNorm is applied before the feature map; learned linear projections are used for $f$5 and $f$6; a data-dependent gate is borrowed from gated linear attention; and $f$7 are learnable per head and per layer. The decay parameters are not fixed. Instead,

$f$8

with sigmoid reparameterization for stability:

$f$9

The paper further notes that the gate is initialized nearly closed so that the physical prior dominates early training. In the decoupling-capacitor placement setup, the appendix reports 8 heads and 3 layers (Yang, 11 May 2026).

The architecture is described as plug-and-play. PSLA replaces the attention layer only; embeddings, feed-forwards, and decoders remain unchanged. It is integrated into different backbones: DevFormer for decoupling-capacitor placement, ChiPFormer for macro placement, and a UNet bottleneck for CircuitNet-based IR-drop prediction (Yang, 11 May 2026). This suggests that the mechanism is intended as a reusable inductive-bias layer rather than a task-specific model redesign.

4. Complexity and efficiency profile

PSLA preserves the asymptotic complexity of standard linear attention. The paper states the following comparison (Yang, 11 May 2026):

Attention type	Time complexity	Memory complexity
Softmax attention	$|Z_{\mathrm{tr}(i,j,f)| \approx Z_0(f)\cdot \exp\!\bigl(-\alpha_x|x_i-x_j| - \alpha_y|y_i-y_j|\bigr)$0	$|Z_{\mathrm{tr}(i,j,f)| \approx Z_0(f)\cdot \exp\!\bigl(-\alpha_x|x_i-x_j| - \alpha_y|y_i-y_j|\bigr)$1
Linear attention	$|Z_{\mathrm{tr}(i,j,f)| \approx Z_0(f)\cdot \exp\!\bigl(-\alpha_x|x_i-x_j| - \alpha_y|y_i-y_j|\bigr)$2	$|Z_{\mathrm{tr}(i,j,f)| \approx Z_0(f)\cdot \exp\!\bigl(-\alpha_x|x_i-x_j| - \alpha_y|y_i-y_j|\bigr)$3
PSLA	$|Z_{\mathrm{tr}(i,j,f)| \approx Z_0(f)\cdot \exp\!\bigl(-\alpha_x|x_i-x_j| - \alpha_y|y_i-y_j|\bigr)$4	$|Z_{\mathrm{tr}(i,j,f)| \approx Z_0(f)\cdot \exp\!\bigl(-\alpha_x|x_i-x_j| - \alpha_y|y_i-y_j|\bigr)$5

The additional Manhattan-decay terms are only elementwise multiplications, so they do not alter the asymptotic linear-attention scaling. The practical gains become pronounced when the sequence is long enough for quadratic attention to dominate runtime and memory.

On the decoupling-capacitor placement encoder at a $|Z_{\mathrm{tr}(i,j,f)| \approx Z_0(f)\cdot \exp\!\bigl(-\alpha_x|x_i-x_j| - \alpha_y|y_i-y_j|\bigr)$6 grid ($|Z_{\mathrm{tr}(i,j,f)| \approx Z_0(f)\cdot \exp\!\bigl(-\alpha_x|x_i-x_j| - \alpha_y|y_i-y_j|\bigr)$7), the reported efficiency gains are about $|Z_{\mathrm{tr}(i,j,f)| \approx Z_0(f)\cdot \exp\!\bigl(-\alpha_x|x_i-x_j| - \alpha_y|y_i-y_j|\bigr)$8 inference speedup and about 98.5% memory savings, with PSLA using about 65 MB. At $|Z_{\mathrm{tr}(i,j,f)| \approx Z_0(f)\cdot \exp\!\bigl(-\alpha_x|x_i-x_j| - \alpha_y|y_i-y_j|\bigr)$9, the reported speedup is about $|Z_{\mathrm{tr}(i,j,f)| \propto \exp\!\bigl(-\kappa(f)\,n\bigr), \qquad n = |x_i - x_j| + |y_i - y_j|.$0, while softmax requires roughly 30 GB and exceeds typical GPU limits. On ChiPFormer, attention-layer memory drops by 94% from 1.2 GB to 68 MB, with modest latency improvement at the tested operating point. On CircuitNet, PSLA adds a small overhead because the bottleneck there is not attention-heavy (Yang, 11 May 2026).

These results delimit the circumstances under which PSLA is most useful. It is not presented as uniformly faster in every architecture; rather, its benefits are largest when attention itself is the computational bottleneck and the sequence length is sufficiently large.

5. Empirical behavior across EDA tasks

PSLA is evaluated in three EDA scenarios: decoupling-capacitor placement, macro placement, and IR-drop prediction (Yang, 11 May 2026). The decoupling-capacitor placement setup uses a DevFormer-style autoregressive decoder on a public $|Z_{\mathrm{tr}(i,j,f)| \propto \exp\!\bigl(-\kappa(f)\,n\bigr), \qquad n = |x_i - x_j| + |y_i - y_j|.$1 dataset and 2,300 generated $|Z_{\mathrm{tr}(i,j,f)| \propto \exp\!\bigl(-\kappa(f)\,n\bigr), \qquad n = |x_i - x_j| + |y_i - y_j|.$2 instances, with supervised imitation learning, REINFORCE online reinforcement learning, and zero-shot cross-scale transfer from $|Z_{\mathrm{tr}(i,j,f)| \propto \exp\!\bigl(-\kappa(f)\,n\bigr), \qquad n = |x_i - x_j| + |y_i - y_j|.$3 to $|Z_{\mathrm{tr}(i,j,f)| \propto \exp\!\bigl(-\kappa(f)\,n\bigr), \qquad n = |x_i - x_j| + |y_i - y_j|.$4. Macro placement uses ChiPFormer or Decision Transformer on ISPD 2005 adaptec1 with 452 macros, under offline decision-transformer pretraining and online reinforcement-learning fine-tuning. IR-drop prediction uses a UNet with PSLA in the bottleneck on CircuitNet 1.0, including the designs Vortex, NVDLA, openc910, zero-riscy, VSmall, and VLarge, evaluated in-distribution, cross-design, and cross-architecture.

The ablation evidence isolates PSLA from PBRS and reveals a pattern rather than a universal advantage. In supervised decoupling-capacitor placement at $|Z_{\mathrm{tr}(i,j,f)| \propto \exp\!\bigl(-\kappa(f)\,n\bigr), \qquad n = |x_i - x_j| + |y_i - y_j|.$5, PSLA achieves $|Z_{\mathrm{tr}(i,j,f)| \propto \exp\!\bigl(-\kappa(f)\,n\bigr), \qquad n = |x_i - x_j| + |y_i - y_j|.$6 versus DevFormer’s $|Z_{\mathrm{tr}(i,j,f)| \propto \exp\!\bigl(-\kappa(f)\,n\bigr), \qquad n = |x_i - x_j| + |y_i - y_j|.$7, a 5.5% improvement. In reinforcement learning, PSLA helps only a little at $|Z_{\mathrm{tr}(i,j,f)| \propto \exp\!\bigl(-\kappa(f)\,n\bigr), \qquad n = |x_i - x_j| + |y_i - y_j|.$8, and with PBRS the gap vanishes; at $|Z_{\mathrm{tr}(i,j,f)| \propto \exp\!\bigl(-\kappa(f)\,n\bigr), \qquad n = |x_i - x_j| + |y_i - y_j|.$9, PSLA provides a large benefit, and with PBRS the best score is $\phi$0. Under zero-shot cross-scale transfer from $\phi$1 to $\phi$2, DevFormer degrades sharply, while PSLA remains much more stable, giving a 56.8% relative gain over DevFormer. In macro placement pretraining, PSLA improves HPWL by up to 12% in decision-transformer pretraining. In CircuitNet, PSLA is slightly worse in-distribution, at $\phi$3 versus $\phi$4, a -1.9% drop, but improves performance by about 5.3–5.4% under cross-design and cross-architecture shift (Yang, 11 May 2026).

The paper summarizes this pattern as a “data-insufficiency principle”: the less data there is to learn spatial structure, the more the physical prior matters. A plausible implication is that PSLA is best understood as a bias-variance intervention. When spatial structure is hard to infer from data alone, the Manhattan-decay prior regularizes the model toward physically plausible interactions. When the dataset is rich enough, that same constraint can reduce flexibility and slightly hurt in-distribution performance.

6. Relation to physics-structured linear attention beyond EDA

The PhysEDA formulation defines PSLA in the specific context of EDA, but the broader idea of expressing physics-guided structure through linear-attention factorization also appears in neural-operator research (Hu et al., 9 Nov 2025). In that work, Transolver’s Physics-Attention is reinterpreted as a special case of linear attention once the slice representation is analyzed carefully. The proposed Linear Attention Neural Operator (LinearNO) then redesigns Physics-Attention into a canonical linear-attention form while preserving the structured bottleneck induced by slicing and deslicing.

The original Physics-Attention pipeline maps grid points to slices, performs attention on slice tokens, and maps the result back to grid-point predictions. The key reinterpretation is that its main functionality can be written in a linear-attention-style factorization with a query-side feature map, an exponential kernel on keys, a value projection, and normalization by summed key scores. The work further interprets LinearNO as a Monte Carlo approximation of a continuous integral operator and proves convergence in probability for the empirical operator (Hu et al., 9 Nov 2025).

The empirical claim in that setting is that the main benefit of Transolver does not come primarily from slice-to-slice attention itself. Instead, the useful component is the slice/deslice bottleneck, which compresses the grid into structured latent tokens and reconstructs back to the grid. The paper states that slice attention may actually hurt performance, and that the structural redesign into a linear-attention operator yields state-of-the-art performance on six standard PDE benchmarks while reducing the number of parameters by an average of 40.0% and computational cost by 36.2%, with additional gains on AirfRANS and Shape-Net Car (Hu et al., 9 Nov 2025).

This suggests a broader usage of the expression “physics-structured linear attention”: a family resemblance among mechanisms that preserve linear-attention algebra while embedding physically meaningful structure into the latent representation or attention kernel. In PhysEDA, the structure is separable Manhattan-distance decay on chip coordinates. In the Transolver reinterpretation, the structure arises from physics-aware slicing/deslicing and integral-operator approximation. The commonality is not a single canonical module, but the conjunction of linear-time factorization and explicit physical inductive bias.

7. Limitations, scope, and interpretation

The main limitation stated for PSLA is that the experimental implementation uses a rank-1 directional approximation rather than the exact symmetric Manhattan kernel (Yang, 11 May 2026). The exact symmetric form is possible through an $\phi$5 bidirectional decomposition described in the appendix, but the faster directional form is used in the main results. This matters because the approximation privileges an efficient separable implementation over exact symmetry.

A second limitation concerns the division of labor between PSLA and PBRS. PBRS applies to online reinforcement learning and does not transfer in the same way to purely supervised settings or offline pretraining. Consequently, improvements attributable to PhysEDA as a whole should not be conflated with improvements attributable specifically to PSLA.

The empirical record also rules out the simplified claim that PSLA is uniformly superior. In CircuitNet in-distribution evaluation, it is slightly worse than baseline, while under distribution shift it improves performance. The evidence therefore supports a narrower interpretation: PSLA is most effective when the task is large-scale, sparse-data, or distribution-shifted, and when the underlying spatial inductive bias is difficult to learn reliably from data alone (Yang, 11 May 2026).

Within that scope, PSLA is best understood as a method for embedding a physically justified decay law into the kernel of linear attention itself. Its defining technical feature is the multiplicative reweighting of query and key feature maps by separable spatial factors derived from normalized chip coordinates and learnable decay rates. Its broader significance lies in showing how domain-specific physical structure can be encoded without abandoning the algorithmic advantages of linear attention.