Papers
Topics
Authors
Recent
Search
2000 character limit reached

GateProbe: Probing Techniques in Diverse Domains

Updated 6 July 2026
  • GateProbe is a set of gate-mediated probing techniques that enable system perturbation analysis across diverse research settings.
  • It is applied in vision-language-action models, scanning gate microscopy, and silicon device probing, each with distinct metrics and observable effects.
  • GateProbe also informs residual-stream interventions, aiding layer selection and transformer interpretability through precise gradient-based sensitivity measures.

Searching arXiv for papers relevant to "GateProbe" and the cited works. Searching arXiv for "GateProbe" and related probing papers. GateProbe is used in several distinct research settings to denote a gate-mediated probe that perturbs or interrogates a system. In Vision-Language-Action model analysis, GateProbe is a one-shot, gradient-based sensitivity metric that inserts a virtual scalar gate on each residual branch and ranks transformer blocks by contribution to downstream action loss within the Drop-Then-Recovery protocol (Sun et al., 26 Jun 2026). In scanning gate microscopy, GateProbe is the use of a conductive, biased tip as a local electrostatic gate that perturbs electron transport in a buried two-dimensional electron gas and enables conductance imaging across strongly and weakly invasive regimes (Steinacher et al., 2017). In silicon electronic devices, GateProbe is a localized single-electron quantum dot induced directly inside the device by the biased tip of a low-temperature scanning tunneling microscope, simultaneously short-range tunnel-coupled to a source reservoir and energy-controlled by a distant gate reservoir (Ng et al., 2020). A related transformer-interpretability literature does not use the name GateProbe, but “Geometric Evolution Maps” provides methodology directly relevant to any gating-based probing approach that operates on concept probes in the residual stream, especially for layer selection, intervention placement, and direction specificity (Henry, 25 May 2026).

1. Terminological scope

The term spans at least three operationally different probe classes and one closely related methodology. What unifies them is the use of a gate or gate-like intervention to expose latent structure; what differs is the substrate, the observable, and the perturbative model.

Usage Gate or probe mechanism Primary observable
VLA model analysis Virtual scalar gate on a residual branch Action-loss sensitivity and post-removal recoverability
Scanning gate microscopy Conductive biased tip creating UtipU_\mathrm{tip} in a 2DEG G(x,y)G(x,y), lcl_\mathrm{c}, and σG\sigma_G
Silicon STM device probing Tip-induced single-electron quantum dot inside the device Resonant tunneling, lever arms, field mapping, defect sensing
Residual-stream concept intervention Settled concept direction chosen after rotation ceases Separation suppression under directional ablation

The resulting literature should not be collapsed into a single definition. In the VLA setting, GateProbe is an activation-space importance metric for structured pruning and recoverability analysis. In mesoscopic transport, it is a local electrostatic perturbation whose weak-limit response can be tied to properties of the unperturbed system. In atomic-scale silicon devices, it is a movable, gate-controllable single-electron spectroscopic element embedded within the device. The residual-stream work suggests a further extension: gate-based interventions can depend critically on where a representation has geometrically settled, rather than merely where a scalar separation score peaks (Sun et al., 26 Jun 2026, Steinacher et al., 2017, Ng et al., 2020, Henry, 25 May 2026).

2. GateProbe as a virtual-gate sensitivity metric in VLA models

In "Drop-Then-Recovery: How Redundant Are Vision-Language-Action Models?" GateProbe is defined as a one-shot, gradient-based sensitivity metric that operates in activation space to quantify each transformer block’s contribution to the downstream action loss. Each droppable transformer block BiB_i is written in residual form,

hi=hi1+Fi(hi1;θi),h_i = h_{i-1} + F_i(h_{i-1};\, \theta_i),

and GateProbe inserts a virtual scalar gate on the residual branch,

h~i=hi1+αiFi(hi1;θi).\tilde{h}_i = h_{i-1} + \alpha_i \cdot F_i(h_{i-1};\, \theta_i).

Setting αi=0\alpha_i = 0 is equivalent to dropping the block, whereas αi=1\alpha_i = 1 recovers the original model.

The importance score is the expected absolute sensitivity of the downstream task loss to that gate,

Igate(Bi)=ExD[L(x)αiαi=1].I_{\text{gate}}(B_i) = \mathbb{E}_{x \sim \mathcal{D}} \left[ \left| \frac{\partial \mathcal{L}(x)}{\partial \alpha_i} \right|_{\alpha_i=1} \right].

By the chain rule,

G(x,y)G(x,y)0

with G(x,y)G(x,y)1. The appendix also writes the layerwise score as

G(x,y)G(x,y)2

The paper interprets this as a first-order Taylor approximation of the loss change when scaling a residual contribution to zero:

G(x,y)G(x,y)3

Larger G(x,y)G(x,y)4 indicates a larger instantaneous loss increase under removal, so the block is deemed more important and is preferentially kept (Sun et al., 26 Jun 2026).

The implementation is explicitly hook-based rather than architectural. No actual parameters are added. Forward pre-hooks at each block’s input normalization layer cache G(x,y)G(x,y)5 and G(x,y)G(x,y)6, and G(x,y)G(x,y)7 is marked with retain_grad(). One forward pass computes G(x,y)G(x,y)8; one backward pass yields G(x,y)G(x,y)9; the algorithm then accumulates

lcl_\mathrm{c}0

over calibration batches, and sets the final score by averaging over lcl_\mathrm{c}1 batches. The ranking criterion is descending importance, with lower-scored blocks removed first. This is presented as Algorithm 1, “Importance Profiling,” and its measured cost on lcl_\mathrm{c}2 with lcl_\mathrm{c}3 calibration batches of batch size lcl_\mathrm{c}4 on one H200 GPU is lcl_\mathrm{c}5, compared with lcl_\mathrm{c}6 for IGIA, lcl_\mathrm{c}7 for Fisher, lcl_\mathrm{c}8 for Hessian trace, lcl_\mathrm{c}9 for CosSim, σG\sigma_G0 for PPL, σG\sigma_G1 for Magnitude, and σG\sigma_G2 for Taylor because of CPU accumulation overhead (Sun et al., 26 Jun 2026).

3. Drop-Then-Recovery, recoverability, and empirical asymmetry

Within Drop-Then-Recovery, GateProbe is used in Stage 1 to select which blocks to remove. Given a target drop count σG\sigma_G3, the method removes the σG\sigma_G4 least important blocks, physically short-circuiting the residual connection σG\sigma_G5 and discarding parameters σG\sigma_G6. Stage 2 then fine-tunes the reduced model against the downstream action objective,

σG\sigma_G7

with recoverability measured by post-recovery Success Rate on evaluation suites such as LIBERO, LIBERO-Plus, and RoboTwin 2.0.

The central empirical result is a strong asymmetry across pathways. Language backbones are described as highly redundant for standard robotic manipulation tasks, whereas vision and action pathways are substantially less tolerant to removal. On LIBERO, removing half of the language blocks improves OpenVLA-OFT from σG\sigma_G8 to σG\sigma_G9 under the same downstream fine-tuning budget, and retaining only two language blocks still recovers baseline-level performance at BiB_i0. For BiB_i1, the corresponding values are BiB_i2 when half the language layers are removed, and BiB_i3 when only two language blocks are kept. By contrast, vision and action compression degrade performance much more severely: on BiB_i4, Vision Drop Half yields BiB_i5, Vision Keep 2 yields BiB_i6, and Action Keep 2 yields BiB_i7. Under FLOPs-matched training on BiB_i8, Drop-12, which keeps BiB_i9 of hi=hi1+Fi(hi1;θi),h_i = h_{i-1} + F_i(h_{i-1};\, \theta_i),0 language blocks, reaches hi=hi1+Fi(hi1;θi),h_i = h_{i-1} + F_i(h_{i-1};\, \theta_i),1, exceeding the hi=hi1+Fi(hi1;θi),h_i = h_{i-1} + F_i(h_{i-1};\, \theta_i),2 baseline; even Drop-17, which keeps hi=hi1+Fi(hi1;θi),h_i = h_{i-1} + F_i(h_{i-1};\, \theta_i),3 block, recovers to hi=hi1+Fi(hi1;θi),h_i = h_{i-1} + F_i(h_{i-1};\, \theta_i),4 (Sun et al., 26 Jun 2026).

Against alternative importance metrics, GateProbe is reported as best or second-best, with its advantage growing under aggressive compression. On hi=hi1+Fi(hi1;θi),h_i = h_{i-1} + F_i(h_{i-1};\, \theta_i),5 across four drop levels, GateProbe averages hi=hi1+Fi(hi1;θi),h_i = h_{i-1} + F_i(h_{i-1};\, \theta_i),6 at Drop-9, hi=hi1+Fi(hi1;θi),h_i = h_{i-1} + F_i(h_{i-1};\, \theta_i),7 at Drop-12, hi=hi1+Fi(hi1;θi),h_i = h_{i-1} + F_i(h_{i-1};\, \theta_i),8 at Drop-16, and hi=hi1+Fi(hi1;θi),h_i = h_{i-1} + F_i(h_{i-1};\, \theta_i),9 at Drop-17. The paper notes that at Drop-16 it exceeds weaker baselines by as much as h~i=hi1+αiFi(hi1;θi).\tilde{h}_i = h_{i-1} + \alpha_i \cdot F_i(h_{i-1};\, \theta_i).0, and at Drop-17 by as much as h~i=hi1+αiFi(hi1;θi).\tilde{h}_i = h_{i-1} + \alpha_i \cdot F_i(h_{i-1};\, \theta_i).1. The comparison set includes parameter-space Taylor, IGIA, Fisher, Hessian trace, activation-space CosSim, PPL, and Magnitude. The stated rationale for preferring GateProbe is that it couples each block’s residual contribution directly to downstream sensitivity, requires only standard backpropagation, and is explicitly aligned with the action-learning loss rather than with static similarity or parameter magnitude (Sun et al., 26 Jun 2026).

The real-robot industrial results qualify the apparent redundancy of language depth. On an xArm 850 warehouse parcel-sorting setup, large language drops preserve near-baseline performance in near-training conditions: in Env 1, Baseline h~i=hi1+αiFi(hi1;θi).\tilde{h}_i = h_{i-1} + \alpha_i \cdot F_i(h_{i-1};\, \theta_i).2, Drop-9 h~i=hi1+αiFi(hi1;θi).\tilde{h}_i = h_{i-1} + \alpha_i \cdot F_i(h_{i-1};\, \theta_i).3, Drop-16 h~i=hi1+αiFi(hi1;θi).\tilde{h}_i = h_{i-1} + \alpha_i \cdot F_i(h_{i-1};\, \theta_i).4; in Env 2, Baseline h~i=hi1+αiFi(hi1;θi).\tilde{h}_i = h_{i-1} + \alpha_i \cdot F_i(h_{i-1};\, \theta_i).5, Drop-9 h~i=hi1+αiFi(hi1;θi).\tilde{h}_i = h_{i-1} + \alpha_i \cdot F_i(h_{i-1};\, \theta_i).6, Drop-16 h~i=hi1+αiFi(hi1;θi).\tilde{h}_i = h_{i-1} + \alpha_i \cdot F_i(h_{i-1};\, \theta_i).7. Under distribution shifts such as lighting changes, novel objects, and container changes, heavily dropped models degrade more than the full model. A common misconception would be to read high recoverability on LIBERO as evidence that language capacity is generally unnecessary; the reported out-of-distribution results instead indicate that redundancy is benchmark- and distribution-dependent. The paper’s limitations also emphasize that GateProbe is a first-order approximation, may miss higher-order block interactions, and can show increased variance with extremely small calibration sets (Sun et al., 26 Jun 2026).

4. GateProbe in scanning gate microscopy

In "Scanning gate experiments: from strongly to weakly invasive probes," GateProbe refers to the use of a conductive, biased tip as a local electrostatic gate that perturbs electron transport in a buried two-dimensional electron gas. The experiment uses an open resonator fabricated in a high-mobility GaAs/AlGaAs 2DEG located h~i=hi1+αiFi(hi1;θi).\tilde{h}_i = h_{i-1} + \alpha_i \cdot F_i(h_{i-1};\, \theta_i).8 below the surface, with density h~i=hi1+αiFi(hi1;θi).\tilde{h}_i = h_{i-1} + \alpha_i \cdot F_i(h_{i-1};\, \theta_i).9, mobility αi=0\alpha_i = 00, and temperature αi=0\alpha_i = 01. A quantum point contact with lithographic gap αi=0\alpha_i = 02 lies at the bottom of the scan area. A semicircular cavity gate of radius αi=0\alpha_i = 03 and opening angle αi=0\alpha_i = 04, centered at the QPC, depletes the 2DEG below it for αi=0\alpha_i = 05. The working point is αi=0\alpha_i = 06, placing the QPC on the third spin-degenerate plateau, αi=0\alpha_i = 07. The tip is scanned αi=0\alpha_i = 08 above the surface and the two-terminal linear conductance map

αi=0\alpha_i = 09

is recorded with αi=1\alpha_i = 10 and lock-in detection.

The tip voltage is calibrated through a dimensionless tip strength,

αi=1\alpha_i = 11

where the least-invasive voltage is αi=1\alpha_i = 12 and the tip begins to deplete the 2DEG at αi=1\alpha_i = 13, so that αi=1\alpha_i = 14 there. The sample has αi=1\alpha_i = 15, and the tip potential is written

αi=1\alpha_i = 16

For αi=1\alpha_i = 17, COMSOL indicates an approximately Lorentzian profile; for αi=1\alpha_i = 18, a Gaussian tail better describes the potential. The appendix uses the Lorentzian example

αi=1\alpha_i = 19

whereas the numerical transport simulations adopt a Gaussian tip with width parameter Igate(Bi)=ExD[L(x)αiαi=1].I_{\text{gate}}(B_i) = \mathbb{E}_{x \sim \mathcal{D}} \left[ \left| \frac{\partial \mathcal{L}(x)}{\partial \alpha_i} \right|_{\alpha_i=1} \right].0 to suppress long-range tails.

The paper’s central distinction is between strongly invasive and weakly invasive GateProbe operation. In the strongly invasive regime, Igate(Bi)=ExD[L(x)αiαi=1].I_{\text{gate}}(B_i) = \mathbb{E}_{x \sim \mathcal{D}} \left[ \left| \frac{\partial \mathcal{L}(x)}{\partial \alpha_i} \right|_{\alpha_i=1} \right].1, the tip induces a depletion disk and acts as a movable hard wall. With Igate(Bi)=ExD[L(x)αiαi=1].I_{\text{gate}}(B_i) = \mathbb{E}_{x \sim \mathcal{D}} \left[ \left| \frac{\partial \mathcal{L}(x)}{\partial \alpha_i} \right|_{\alpha_i=1} \right].2, branched electron flow appears as strong thin features in Igate(Bi)=ExD[L(x)αiαi=1].I_{\text{gate}}(B_i) = \mathbb{E}_{x \sim \mathcal{D}} \left[ \left| \frac{\partial \mathcal{L}(x)}{\partial \alpha_i} \right|_{\alpha_i=1} \right].3 maps; once the cavity is formed, branch patterns disappear and conductance modulations spread over the cavity area with characteristic Igate(Bi)=ExD[L(x)αiαi=1].I_{\text{gate}}(B_i) = \mathbb{E}_{x \sim \mathcal{D}} \left[ \left| \frac{\partial \mathcal{L}(x)}{\partial \alpha_i} \right|_{\alpha_i=1} \right].4 scale. In the weakly invasive regime, Igate(Bi)=ExD[L(x)αiαi=1].I_{\text{gate}}(B_i) = \mathbb{E}_{x \sim \mathcal{D}} \left[ \left| \frac{\partial \mathcal{L}(x)}{\partial \alpha_i} \right|_{\alpha_i=1} \right].5, the tip cannot efficiently backscatter through the QPC when Igate(Bi)=ExD[L(x)αiαi=1].I_{\text{gate}}(B_i) = \mathbb{E}_{x \sim \mathcal{D}} \left[ \left| \frac{\partial \mathcal{L}(x)}{\partial \alpha_i} \right|_{\alpha_i=1} \right].6, and once the cavity is formed the map develops rich spatial structures inside the cavity region through gentle lensing rather than hard reflections. Near Igate(Bi)=ExD[L(x)αiαi=1].I_{\text{gate}}(B_i) = \mathbb{E}_{x \sim \mathcal{D}} \left[ \left| \frac{\partial \mathcal{L}(x)}{\partial \alpha_i} \right|_{\alpha_i=1} \right].7, the spatial pattern becomes insensitive to tip strength while the amplitude scales linearly with Igate(Bi)=ExD[L(x)αiαi=1].I_{\text{gate}}(B_i) = \mathbb{E}_{x \sim \mathcal{D}} \left[ \left| \frac{\partial \mathcal{L}(x)}{\partial \alpha_i} \right|_{\alpha_i=1} \right].8; the paper presents this as the hallmark of the perturbative regime.

The perturbative theory uses the first-order conductance correction

Igate(Bi)=ExD[L(x)αiαi=1].I_{\text{gate}}(B_i) = \mathbb{E}_{x \sim \mathcal{D}} \left[ \left| \frac{\partial \mathcal{L}(x)}{\partial \alpha_i} \right|_{\alpha_i=1} \right].9

with

G(x,y)G(x,y)00

and the standard Landauer expression

G(x,y)G(x,y)01

Within the weak regime, the amplitude of the SGM response scales linearly with G(x,y)G(x,y)02, and a correlation analysis in simulation shows that the full response remains accurately predicted by first order up to G(x,y)G(x,y)03, corresponding through calibration to G(x,y)G(x,y)04. Inside this window, changing G(x,y)G(x,y)05 rescales the amplitude without modifying the spatial pattern, so G(x,y)G(x,y)06 is determined by the unperturbed reflection and transmission matrices and the unperturbed scattering states.

The paper quantifies invasiveness using a characteristic length scale G(x,y)G(x,y)07 and a modulation amplitude G(x,y)G(x,y)08. From the one-dimensional power spectral density

G(x,y)G(x,y)09

it fits

G(x,y)G(x,y)10

defines

G(x,y)G(x,y)11

and finds that G(x,y)G(x,y)12 peaks near G(x,y)G(x,y)13 at approximately G(x,y)G(x,y)14 before decreasing toward G(x,y)G(x,y)15, with measured G(x,y)G(x,y)16. The perturbative regime is thus characterized by maximal G(x,y)G(x,y)17, minimal G(x,y)G(x,y)18, and linear amplitude scaling. The maximum G(x,y)G(x,y)19 is disorder-limited and comparable to the disorder correlation length G(x,y)G(x,y)20. The practical guidance is correspondingly explicit: identify G(x,y)G(x,y)21 from minimal G(x,y)G(x,y)22 and maximal G(x,y)G(x,y)23, operate within G(x,y)G(x,y)24, avoid G(x,y)G(x,y)25 if an interpretation in terms of the unperturbed system is desired, and treat large positive G(x,y)G(x,y)26 with caution because screening charges in the doping plane complicate the response (Steinacher et al., 2017).

5. GateProbe as a tip-induced single-electron probe in silicon

In "Scanned single-electron probe inside a silicon electronic device," GateProbe is realized by inducing a localized quantum dot directly inside a working multi-terminal silicon device with the biased tip of a low-temperature STM. A negative sample bias G(x,y)G(x,y)27 bends the silicon bands downward locally under the tip and forms a confinement potential that traps an electron beneath the hydrogen-terminated silicon surface. The dot is centered under the tip and therefore follows the tip laterally with sub-nanometer precision. The source reservoir is an Sb-doped G(x,y)G(x,y)28 region that supplies electrons to the dot at rate G(x,y)G(x,y)29 when the dot level aligns with the source chemical potential; the dot then empties to the tip at rate G(x,y)G(x,y)30. A separate Sb-doped G(x,y)G(x,y)31 gate reservoir, approximately G(x,y)G(x,y)32 away across an undoped/p-type channel, shifts the dot level electrostatically via G(x,y)G(x,y)33.

The transport signature is resonant single-electron tunneling. A resonant step in G(x,y)G(x,y)34, or peak in G(x,y)G(x,y)35, appears when the dot level aligns with the source reservoir, and the current through a single discrete level is

G(x,y)G(x,y)36

The experiment is operated in the regime G(x,y)G(x,y)37, so the measured peak height primarily reflects G(x,y)G(x,y)38. This was verified by varying the tip height by G(x,y)G(x,y)39–G(x,y)G(x,y)40 at fixed lateral position, which left the peak height nearly unchanged. As the tip is moved toward the source by only a few nanometers, the QD–source separation decreases and G(x,y)G(x,y)41 increases exponentially. The measured peak-current increase between G(x,y)G(x,y)42 and G(x,y)G(x,y)43 follows

G(x,y)G(x,y)44

with best-fit G(x,y)G(x,y)45. Using the overlap model described in the supplementary material gives a QD wavefunction decay length

G(x,y)G(x,y)46

The WKB expression

G(x,y)G(x,y)47

is used as the tunneling picture behind the effective position-dependent barrier (Ng et al., 2020).

The electrostatics are expressed through capacitances G(x,y)G(x,y)48, G(x,y)G(x,y)49, G(x,y)G(x,y)50, and G(x,y)G(x,y)51, with total

G(x,y)G(x,y)52

and lever arms G(x,y)G(x,y)53. The dot level shifts as

G(x,y)G(x,y)54

In the STM biasing configuration, the effective bias lever arm is

G(x,y)G(x,y)55

From the spatially resolved spectroscopy, the dot resonance in G(x,y)G(x,y)56 shifts by approximately G(x,y)G(x,y)57 as G(x,y)G(x,y)58 increases by G(x,y)G(x,y)59, and field-based calibration yields G(x,y)G(x,y)60. Independent fits to a single-level tunneling model give G(x,y)G(x,y)61 as G(x,y)G(x,y)62 increases from G(x,y)G(x,y)63 to G(x,y)G(x,y)64. In the stability diagram, the resonance slope is G(x,y)G(x,y)65 for G(x,y)G(x,y)66, which with G(x,y)G(x,y)67 gives a gate lever arm

G(x,y)G(x,y)68

The capacitance ratios extracted in the supplementary material are

G(x,y)G(x,y)69

These values show that the source dominates the dot’s self-capacitance, but the lithographic gate still provides substantial capacitive control even though the dot is only approximately G(x,y)G(x,y)70 from the metallic tip.

The GateProbe is also used as a local electrometer. During spatial scans with G(x,y)G(x,y)71 and G(x,y)G(x,y)72 near resonance, the planar electric field between source and gate is estimated as

G(x,y)G(x,y)73

giving approximately G(x,y)G(x,y)74. More generally, once G(x,y)G(x,y)75 is calibrated, local electric fields can be mapped from resonance shifts through

G(x,y)G(x,y)76

When the moving dot passes near localized charges, its level shifts by Coulomb interaction, producing spatially localized distortions of the resonance. The reported examples include a step edge and a doubly charged dangling bond; approaching a negative charge requires more negative G(x,y)G(x,y)77 to maintain resonance, allowing both defect polarity and location to be identified with atomic-scale precision.

The paper positions this GateProbe against scanning single-electron transistor microscopy and scanning gate microscopy. Unlike those approaches, where interaction is purely capacitive and long-ranged, the silicon GateProbe achieves short-range tunnel coupling between the probe dot and the device source reservoir while retaining sizable gate control, with G(x,y)G(x,y)78 and G(x,y)G(x,y)79. The limitations are equally explicit: the probe necessarily perturbs the local potential through tip-induced band bending, can hybridize with localized states in heavily doped regions, and requires cryogenic temperature and UHV. Within those constraints, it provides atomic-scale spatial resolution, quantitative tunnel-coupling control, and in-device electrometry (Ng et al., 2020).

6. Relation to residual-stream concept probing: Geometric Evolution Maps

"Geometric Evolution Maps: Extracting Stable Concept Probes from Transformer Residual Streams" does not mention GateProbe by name. The paper is nevertheless directly relevant to any gating-based probing approach that operates on concept probes in the residual stream, including layer selection, ablation or intervention placement, and direction specificity. Its starting point is that a concept probe is a direction vector in residual-stream activation space that discriminates between sentences expressing a concept versus matched sentences not expressing that concept. The probe at layer G(x,y)G(x,y)80 is the L2-normalized difference between class centroids,

G(x,y)G(x,y)81

using last-token activations from the post-MLP residual output. The Fisher-normalized separation score is

G(x,y)G(x,y)82

The paper defines a Concept Allocation Zone as the contiguous layer interval containing the primary separation event, from above-noise onset through the peak and back toward a floor.

The key geometric claim is that concept directions rotate substantially during assembly and do not settle into a stable direction until a characteristic handoff layer after the CAZ. Rotation is quantified via adjacent-layer directional similarity

G(x,y)G(x,y)83

and angular velocity

G(x,y)G(x,y)84

Entry-to-exit cosine within the CAZ is

G(x,y)G(x,y)85

Across G(x,y)G(x,y)86 architectures and G(x,y)G(x,y)87 concepts, mean EEC is G(x,y)G(x,y)88 and median EEC is G(x,y)G(x,y)89; G(x,y)G(x,y)90 of pairs have EEC G(x,y)G(x,y)91, and G(x,y)G(x,y)92 have EEC G(x,y)G(x,y)93. The paper interprets this as showing that directions at CAZ entry are poor predictors of settled directions at CAZ exit. The handoff layer is defined as the first post-CAZ layer after rotation ceases under the threshold G(x,y)G(x,y)94:

G(x,y)G(x,y)95

The handoff cosine G(x,y)G(x,y)96 has mean G(x,y)G(x,y)97 and median G(x,y)G(x,y)98, which the paper presents as evidence that the settled probe direction is typically stable through to the final layer (Henry, 25 May 2026).

For intervention, the paper uses directional ablation at the extraction layer,

G(x,y)G(x,y)99

and measures retained percentage of separation. GEM compares the probe extracted at the handoff layer against the probe extracted at the peak layer. Across lcl_\mathrm{c}00 concept-model pairs, GEM-extracted probes are at least as precise as peak-layer probes in lcl_\mathrm{c}01 trials lcl_\mathrm{c}02 and strictly outperform in lcl_\mathrm{c}03 lcl_\mathrm{c}04. Trial-level Wilcoxon on lcl_\mathrm{c}05 non-ties gives lcl_\mathrm{c}06, and model-level Wilcoxon signed-rank gives lcl_\mathrm{c}07, lcl_\mathrm{c}08, lcl_\mathrm{c}09 one-sided; excluding gpt2 yields lcl_\mathrm{c}10, lcl_\mathrm{c}11. The architecture split is pronounced: MHA models favor the handoff in lcl_\mathrm{c}12 trials lcl_\mathrm{c}13, whereas GQA models favor the handoff in lcl_\mathrm{c}14 trials lcl_\mathrm{c}15.

The paper also gives practical guidance that maps directly onto GateProbe-style intervention design. The default ablation width is lcl_\mathrm{c}16 consecutive layers starting at lcl_\mathrm{c}17, but if lcl_\mathrm{c}18 the near-final rule sets lcl_\mathrm{c}19 to avoid contamination by unembedding-preparation layers. This rule is triggered in lcl_\mathrm{c}20 cases lcl_\mathrm{c}21 and improves probe quality in lcl_\mathrm{c}22 triggered cases lcl_\mathrm{c}23 with mean gain lcl_\mathrm{c}24 percentage points; the overall mean across all lcl_\mathrm{c}25 pairs is lcl_\mathrm{c}26 points. A direction-specificity control compares concept-direction ablation against lcl_\mathrm{c}27 random unit vectors per pair, giving mean concept-direction reduction lcl_\mathrm{c}28 versus mean random-direction reduction lcl_\mathrm{c}29, median specificity ratio lcl_\mathrm{c}30, median z-score lcl_\mathrm{c}31, and lcl_\mathrm{c}32 pairs lcl_\mathrm{c}33 in which the concept direction beats all lcl_\mathrm{c}34 random seeds. The practical implication is narrow but important: gate-based probing in residual streams should target a settled direction at the handoff layer rather than a rotating peak-layer direction or an arbitrarily late layer. The stated caveats are that lcl_\mathrm{c}35 is empirically set without a formal sensitivity analysis, late handoffs are common at lcl_\mathrm{c}36, shallow models can fail near the unembedding, some concepts may never settle, and sliding-window attention variants can violate depth-matched expectations (Henry, 25 May 2026).

Topic to Video (Beta)

No one has generated a video about this topic yet.

Whiteboard

No one has generated a whiteboard explanation for this topic yet.

Follow Topic

Get notified by email when new papers are published related to GateProbe.