Disentangle Necessity versus Propensity in CoT Monitorability
Characterize the proportion of observed chain-of-thought monitorability attributable to the necessity of externalizing reasoning versus the natural propensity of models to externalize reasoning across the tasks considered for detecting misbehavior and alignment signals.
References
It is unclear what proportion of the CoT monitorability demonstrated in these examples is due to the necessity versus the propensity for a model to reason out loud in the tasks considered.
The mechanisms underlying these failures remain an open question and we discuss several possible explanations in Appendix \ref{app:hypotheses explaining monitor failures}.
We are excited for future work to measure both the necessity level of dangerous misbehaviours, as well as the monitorability at these higher necessity levels.