Robustness of neutral internal safety signals to monitor-aware attacks

Determine whether the safety signal obtained from neutral-context residual activations remains effective when an attacker knows that the signal is being monitored and adapts its behavior accordingly.

Background

The APPS experiments find that neutral-context activation probes can identify backdoored programs, but the evaluation uses stored attacks that do not adapt to the monitor. The paper explicitly distinguishes this bounded result from robustness against an attacker that knows which internal channel is being read.

The unresolved issue is whether the observed monitoring advantage survives strategic adaptation: an attacker might alter behavior, telemetry, or code specifically to evade the neutral residual signal. Addressing this would require monitor-aware or adaptive attack trajectories rather than fixed offline examples.

References

It shows that a useful signal can appear before the model is asked for a verdict. It does not show that the same signal survives once an attacker knows it is being read.

— ObserverBench: Testing Mechanistic Estimates for Intervention and Control  (2609.03026 - Erramilli, 2 Sep 2026) in Section 4, subsection “Monitor rankings depend on the model and operating regime”