Scaling SIAB to much larger language models

Determine whether the internal attention signal used by Self-targeted Internal Attention Boosting (SIAB) continues to generalize to much larger language models, such as models with 70 billion or more parameters.

Background

The study evaluates SIAB only on Gemma3, Qwen3, and LLaMA3 models with 4–8 billion parameters. Its head localization procedure relies on exhaustive activation patching, which the authors identify as computationally prohibitive at substantially larger scales. Consequently, it remains unresolved whether the internal affective-routing signal identified in these smaller models persists in models with 70 billion or more parameters.