Activation Injection in Neural & Photonic Systems

• Activation injection is the deliberate introduction or modification of internal activation states in digital and analog systems, enabling latent capabilities.
• It encompasses methods like residual steering in LLMs, stochastic noise during training, and optical current injection in photonic hardware, each tailored to specific applications.
• These strategies provide targeted control over model behavior, improve benchmark performance, and aid in defense and interpretability across diverse domains.

Activation injection is a term used across several arXiv literatures to denote the deliberate introduction, modification, or operational use of internal activations or activation-like states. In LLMs, it most often refers to inference-time interventions on residual-stream activations, auxiliary activation-conditioned models, or white-box activation monitoring; in optical and optoelectronic neural hardware, it denotes injected optical or electrical signals that realize nonlinear activation functions; in training literature, it includes stochastic perturbations introduced before or after nonlinearities; and in adjacent physics it denotes activation of collective or global states by injected currents or plasma (Turner et al., 2023, Crnjanski et al., 2023, Gulcehre et al., 2016, Nakamura et al., 2018).

1. Terminological scope

Across the cited literature, the phrase does not denote a single standardized operator. Some papers use it for additive steering of hidden states during inference, some for feeding one model’s activations into another model, some for pre-output monitoring of internal states, and some for physical injection mechanisms that generate nonlinear transfer curves in photonic hardware (Yu et al., 11 Oct 2025, Bauer et al., 23 May 2026, Chauhan et al., 2 Jun 2026, Arahata et al., 8 Jan 2026).

Domain	Meaning of “activation injection”	Representative papers
LLM control	Add or subtract a direction in activation space during inference	(Turner et al., 2023, Wang et al., 2023, Shah et al., 27 May 2026)
Interpretability and defense	Read, compare, or inject activations into auxiliary probes or oracles	(Abdelnabi et al., 2024, Bauer et al., 23 May 2026, Chauhan et al., 2 Jun 2026)
Optical and photonic neural hardware	Use optical injection or current injection to realize nonlinear activation functions	(Crnjanski et al., 2023, Liu et al., 2023, Guo et al., 1 Jul 2025, Arahata et al., 8 Jan 2026)
Training with stochastic perturbations	Inject noise before or after activation functions	(Gulcehre et al., 2016, Maksimov et al., 9 Apr 2026)
Adjacent physics	Activate a mode or magnetosphere by injected current or plasma	(Nakamura et al., 2018, Niv et al., 2023)

A narrower clarification appears in work on direct prompt injection: there, “activation-guided” refers to using surrogate-model hidden activations to guide prompt optimization, and the authors explicitly state that the method does not inject internal tensors into the victim model (Li et al., 9 Sep 2025). This suggests that the phrase is best interpreted locally, by paper and domain, rather than as a universal technical term.

2. Inference-time activation control in LLMs

In the modern LLM literature, the canonical form of activation injection is residual-stream steering. Activation Addition computes a steering vector from a contrast pair, $\mathbf{h}_A^l = \mathbf{h}_+^l - \mathbf{h}_-^l$, and injects it at layer $l$ by $$\mathbf{h}^{l\prime} = \mathbf{h}^l + c\,\mathbf{h}_A^l$$ (Turner et al., 2023). This formulation is explicitly inference-time, uses only forward passes, and is reported to steer topic, sentiment, and style while preserving off-target factual performance on ConceptNet. The same paper reports that middle layers are typically most effective, that coefficients in the range $3$ to $15$ are typical, and that excessively large coefficients can damage syntax (Turner et al., 2023).

Subsequent work generalizes this fixed-vector paradigm in several directions. Trojan Activation Attack computes a trojan steering vector from the mean activation difference between a safety-aligned target model and a less-aligned teacher model, selects the intervention layer by Jensen–Shannon divergence, and injects $\tilde{h}_l = h_l + \lambda s_l$ to bypass refusal on truthfulness, toxicity, bias, and harmfulness benchmarks (Wang et al., 2023). Fusion Steering replaces a single global vector with prompt-specific, layerwise deltas derived from a reference completion that concatenates the ground-truth answer with a generated explanation; on 260 SimpleQA prompts where the baseline failed, segmented steering on Gemma-2-2B-IT reaches 25.4% accuracy under the paper’s composite criterion, compared with 16.2% for full-layer steering and 3.5% for the baseline, and raises fully correct SimpleQA responses from 0.0% to 13.1% (Chang et al., 28 May 2025).

More structured steering methods replace scalar tuning with geometric or task-specific calibration. PIXEL learns a property-aligned subspace from tail-averaged and end-token views, then computes a closed-form minimal intervention strength for each injection site and position, using $$h'_{\ell,t} = h_{\ell,t} + \beta^\ast \tilde{w}_\ell(x)$$ (Yu et al., 11 Oct 2025). On truthfulness, bias, refusal, and helpfulness evaluations, it improves attribute alignment while preserving or improving base-task scores on RACE, MMLU, OpenBookQA, and GLUE (Yu et al., 11 Oct 2025). In a different setting, Activation Consistency Training yields a single learned direction at the assistant-turn boundary; adding that direction to the undefended base model induces refusal on jailbreaks, while subtracting it from the defended model restores compliance. On Qwen3-8B, the steering effect crosses the defended refusal curve at $\alpha \approx 5$ and reaches almost universal refusal at $\alpha = 10$, with benign false refusals remaining near zero up to $\alpha = 5$ (Shah et al., 27 May 2026).

The literature also distinguishes between single-vector and multi-vector control. A study on broad skills and multiple behaviours reports that combining several steering vectors into one vector is largely unsuccessful, whereas injecting individual steering vectors at different layers simultaneously is promising and incurs only a small alignment tax on general text accuracy (Weij et al., 2024). This is important because it frames activation injection not only as a single-direction intervention, but also as a compositional control problem over layers and behaviors.

3. Monitoring, probing, and activation-conditioned auxiliary models

A separate line of work uses activation access for monitoring rather than for steering. In credential-exfiltration defense, activation instrumentation is applied before any output token is emitted. The method registers forward hooks on approximately the last quarter of transformer layers, pools hidden states over readout positions that can attend to both secret context and attacker instructions, computes per-layer Mahalanobis deviations from a benign baseline, and feeds the resulting Causal Flow Score to a probe (Chauhan et al., 2 Jun 2026). On Qwen-2.5-7B in a held-out single-turn mix, this pre-output detector achieves AUROC $l$0, F1 $l$1, and FPR $l$2, with reported overhead of approximately $l$3 ms; when fused with honeytoken detection and multi-turn leakage accounting, the full system reaches detect $l$4 with FPR $l$5 on the held-out single-turn subset (Chauhan et al., 2 Jun 2026).

Task-drift detection adopts a related but distinct formalism: it computes activation deltas between the primary-task prompt and the same prompt after appending external data, pools at the last token, and classifies clean versus poisoned states using either logistic regression or metric learning (Abdelnabi et al., 2024). On out-of-distribution prompt-injection, jailbreak, and malicious-instruction tests, linear probes on late or late-middle layers achieve near-perfect ROC AUC, and the associated TaskTracker release includes more than 500K instances, raw activations from several models, and probe tooling (Abdelnabi et al., 2024). By contrast, activation-guided prompt injection uses surrogate activations to score prompt candidates for black-box attacks; the method reports 49.6% cross-model ASR and 36.6% ASR on unseen task scenarios, but explicitly frames this as activation-guided prompt injection rather than literal activation-level modification of the victim model (Li et al., 9 Sep 2025).

Activation Oracles represent yet another interface. Here, selected residual-stream vectors from a base model are added into an oracle model’s residual stream after the second transformer layer using an additive, norm-matched rule, with $l$6 and $l$7 (Bauer et al., 23 May 2026). Increasing the injection strength, feeding multiple positions, and feeding multiple layers improves AObench from +0.244 in the baseline recipe to +0.435 in the final recipe; the same paper reports a vagueness score increase from 0.076 to 0.205 and an aggregate hallucination metric improvement from 68.8% to 84.6% (Bauer et al., 23 May 2026). In this setting, activation injection is not a control primitive over the base model itself, but a conditioning mechanism that allows a second model to answer questions about another model’s internal state.

4. Optical and optoelectronic realizations of activation functions

In photonic neural hardware, activation injection is often literal optical or electrical injection into a nonlinear device. One implementation uses a bistable Fabry–Pérot semiconductor laser under optical injection as an all-optical activation unit for multilayer perceptrons (Crnjanski et al., 2023). The injected pulse peak power $l$8 is mapped to output peak power $l$9 by a tunable sigmoid-like transfer $$\mathbf{h}^{l\prime} = \mathbf{h}^l + c\,\mathbf{h}_A^l$$0, with the detuning $$\mathbf{h}^{l\prime} = \mathbf{h}^l + c\,\mathbf{h}_A^l$$1 acting as the main control parameter. For bias $$\mathbf{h}^{l\prime} = \mathbf{h}^l + c\,\mathbf{h}_A^l$$2 and FWHM $$\mathbf{h}^{l\prime} = \mathbf{h}^l + c\,\mathbf{h}_A^l$$3 ps, varying $$\mathbf{h}^{l\prime} = \mathbf{h}^l + c\,\mathbf{h}_A^l$$4 from $$\mathbf{h}^{l\prime} = \mathbf{h}^l + c\,\mathbf{h}_A^l$$5 to $$\mathbf{h}^{l\prime} = \mathbf{h}^l + c\,\mathbf{h}_A^l$$6 shifts the activation threshold from below 1 mW to above 70 mW. In the reported two-layer MLP experiments, the maximum averaged test accuracy reaches 95.84% on MNIST and 85.3% on Fashion-MNIST, and tuning detuning pairs improves accuracy by up to approximately 2% relative to suboptimal settings (Crnjanski et al., 2023).

A second optical line realizes ReLU-like and ELU-like nonlinearities above a Hopf bifurcation in an optically injected semiconductor laser. The static regime produces a kinked ReLU-like transfer at positive detuning, with the threshold increasing from approximately 0.15 mW at $$\mathbf{h}^{l\prime} = \mathbf{h}^l + c\,\mathbf{h}_A^l$$7 GHz to approximately 1.7 mW at $$\mathbf{h}^{l\prime} = \mathbf{h}^l + c\,\mathbf{h}_A^l$$8 GHz, while the below-threshold slope decreases from 2.55 to 0.32 and the above-threshold slope remains near 0.065 (Liu et al., 2023). On MNIST, the piecewise-linear abstractions of this transfer reach up to 98.02% accuracy, close to a standard leaky ReLU baseline at 98.08% (Liu et al., 2023). A later broadband implementation moves to the unstable-locked regime above the Hopf bifurcation and demonstrates operation up to 20 GHz: the optical activation mimics a ReLU below 5 GHz and an ELU above 5 GHz, with the 20 GHz regime restored at 60 mA when the relaxation-oscillation frequency reaches approximately 19.7 GHz (Guo et al., 1 Jul 2025). In that work, a three-hidden-layer fully connected network on MNIST achieves 97.9% with the ReLU-like optical activation and 96.1% at 20 GHz with the ELU-like regime (Guo et al., 1 Jul 2025).

Silicon photonic optoelectronic converters provide a related but electrically mediated usage. In the current-injection variant, a germanium photodetector feeds photocurrent directly into a forward-biased microring modulator, and the authors explicitly call the carrier-injection mechanism “Activation Injection” (Arahata et al., 8 Jan 2026). The injection mode has measured resonance-shift efficiency $$\mathbf{h}^{l\prime} = \mathbf{h}^l + c\,\mathbf{h}_A^l$$9 nm/V, compared with $3$0 pm/V in depletion mode; it reaches gain $3$1 at approximately 1 mW optical bias power, exhibits activation slope $3$2 near the operating point, and regenerates extinction ratio from 2.8 dB at the input to 6.7 dB at the output, ანუ 3.9 dB improvement (Arahata et al., 8 Jan 2026). Its measured 3 dB bandwidth is 76 MHz, whereas the load-resistor variant reaches 1.49 GHz and clear eye openings up to 4 Gb/s (Arahata et al., 8 Jan 2026).

5. Noise injection relative to activation functions

Another established meaning of activation injection concerns stochastic perturbations introduced at or around nonlinearities during training. Noisy activation functions target saturating regimes by defining $3$3 and a learned noise scale $3$4, then using a noisy activation such as $3$5 (Gulcehre et al., 2016). The central claim is that gradients in saturated regions become non-zero almost surely through the stochastic term, and that annealing the noise connects the method to simulated annealing and continuation. The reported empirical gains include Penn Treebank word-level LSTM perplexity improving from 119.4/115.6 valid/test in the reference model to 111.7/108.0 with NAN, Europarl NMT BLEU improving from 20.18 to 22.57 with noisy tanh/sigmoid, and the unique-elements counting task dropping test error from 14.83% with curriculum only to 9.53% with annealed noisy activations (Gulcehre et al., 2016).

A more hardware-motivated study examines internal Gaussian noise injected before or after the activation function, in additive and multiplicative forms (Maksimov et al., 9 Apr 2026). The reported conclusion is that the activation function acts as a nonlinear filter: injecting noise before the activation consistently yields higher accuracy than injecting after it, and post-activation multiplicative noise is less harmful than post-activation additive noise. In a one-hidden-layer MNIST network with post-activation noise, accuracy remains near baseline for additive noise up to about $3$6, then falls to approximately 31.6% at $3$7; for multiplicative noise, degradation begins near $3$8 and accuracy remains approximately 47.8% at $3$9 (Maksimov et al., 9 Apr 2026). The same paper reports that earlier hidden layers are more damaging injection sites because downstream weight statistics amplify noise cumulatively, that wider networks are more robust, and that pooling-based noise reduction improves performance in both pre- and post-activation settings (Maksimov et al., 9 Apr 2026).

6. Broader scientific usages and conceptual commonalities

Outside neural computation, the phrase appears in more literal physical senses. In superconducting NbN, dc supercurrent injection activates the Higgs mode in linear infrared response by introducing a finite gauge-invariant condensate momentum; the experiment observes a sharp conductivity peak at 5.40 ± 0.05 meV, with FWHM 0.80 ± 0.05 meV and amplitude up to about 1% of $15$0, only for probe polarization parallel to the current, and with peak amplitude scaling quadratically with injected current (Nakamura et al., 2018). In black-hole magnetospheres, local plasma injection activates or screens the Blandford–Znajek state only if the plasma supply fills the causal region between the inner and outer light surfaces; without pair production, external injection beyond the outer light surface yields either large vacuum gaps or coherent oscillations, whereas enabling pair production with $15$1 produces nearly complete screening and outgoing Poynting flux near $15$2, with a few percent of the power emitted as TeV $15$3-rays (Niv et al., 2023).

Across these otherwise disparate domains, a common structural pattern recurs. Injection introduces an internal degree of freedom that the unperturbed system either lacks or cannot stably exploit: a residual-space direction in an LLM, an auxiliary conditioning channel in an oracle, a detuned optical field or photocurrent in a photonic neuron, stochastic perturbations that reopen gradients, a supercurrent that creates a linear coupling to the Higgs mode, or plasma that screens a magnetosphere (Wang et al., 2023, Bauer et al., 23 May 2026, Crnjanski et al., 2023, Gulcehre et al., 2016, Nakamura et al., 2018, Niv et al., 2023). A plausible implication is that “activation injection” functions less as a single method name than as a family of intervention strategies centered on altering internal state so that a latent capability, response mode, or operating regime becomes accessible.