Predictive mapping from training procedures to cognitive capabilities

Determine predictive relationships between specific large language model training procedures (e.g., supervised fine-tuning, process supervision, reinforcement learning) and the emergence of particular reasoning capabilities (e.g., meta-cognitive monitoring, representational restructuring) so that one can know a priori which training produces which cognitive capabilities.

Background

The paper introduces a taxonomy of 28 cognitive elements and analyzes 171K model reasoning traces to paper how these elements manifest and correlate with success. Despite extensive empirical comparisons and interventions via test-time reasoning guidance, the authors emphasize that there is currently no principled way to predict which training setup yields which cognitive capabilities.

They note strong variations across training paradigms and model families and argue that moving from post-hoc observation to theory-driven experimentation requires establishing predictive mappings from training procedures to emergent cognitive elements.