Learning to Think from Multiple Thinkers

Abstract: We study learning with Chain-of-Thought (CoT) supervision from multiple thinkers, all of whom provide correct but possibly systematically different solutions, e.g., step-by-step solutions to math problems written by different thinkers, or step-by-step execution traces of different programs solving the same problem. We consider classes that are computationally easy to learn using CoT supervision from a single thinker, but hard to learn with only end-result supervision, i.e., without CoT (Joshi et al. 2025). We establish that, under cryptographic assumptions, learning can be hard from CoT supervision provided by two or a few different thinkers, in passive data-collection settings. On the other hand, we provide a generic computationally efficient active learning algorithm that learns with a small amount of CoT data per thinker that is completely independent of the target accuracy $\varepsilon$, a moderate number of thinkers that scales as $\log \frac{1}{\varepsilon}\log \log \frac{1}{\varepsilon}$, and sufficient passive end-result data that scales as $\frac{1}{\varepsilon}\cdot poly\log\frac{1}{\varepsilon}$.

• The paper establishes that learning from multiple thinkers' Chain-of-Thought supervision can become computationally intractable under specific cryptographic and complexity assumptions.
• It introduces an autoregressive linear threshold model that simulates depth-2 threshold circuits, significantly reducing required thinking steps while preserving intermediate reasoning.
• An active, adaptive query protocol using boosting demonstrates that efficient learning is achievable with optimal sample complexity when supervised by diverse thinkers.

Computational Barriers and Algorithmic Pathways in Learning from Multiple Thinkers’ Chains of Thought

Introduction and Framework

The paper "Learning to Think from Multiple Thinkers" (2604.24737) develops a computational and information-theoretic analysis of learning from Chain-of-Thought (CoT) supervision provided by multiple, systematically distinct, but correct "thinkers." This scenario reflects practical realities in LLM training pipelines—e.g., step-by-step mathematical solutions or reasoning demonstrations sourced from disparate human or programmatic agents—where full CoT traces cannot, in practice, be tied to a singular underlying strategy.

Prior work [malach2023auto; joshi2025theory] established the universality and tractability of learning complex autoregressive procedures with full CoT supervision from a single generator, especially when the base class admits a tractable “consistent” oracle (e.g., linear threshold functions). In contrast, purely end-to-end supervision, even with simple base generators, introduces computational intractability due to the compounded expressiveness of autoregressive reasoning. The present work systematically charts the landscape between these extremes for the multi-thinker case, precisely formalizing models where data arises from various thinkers—selected via adversarial, instance-dependent, or random routes—and the learner may (or may not) identify the source thinker for each trace.

Autoregressive Linear Thresholds: Setup and Expressivity

Central to the analysis is the autoregressive linear threshold (AR-LT) model: each base generator is an LTF with window size $d$, iteratively applied to generate new tokens (see Figure 1).

Figure 1: A visual help on how a linear threshold is applied autoregressively.

The key technical insight is an improved simulation of depth-2 threshold circuits by AR-LTs: the paper shows that any such circuit of size $s$ over $n$ inputs can be simulated by an AR-LT with $d = \Theta(ns)$ and $T \leq 2s$ autoregressive steps, significantly reducing the required thinking length relative to earlier constructions [joshi2025theory]. The construction is carefully CoT-preserving: the AR-LT’s intermediate states reproduce (up to mapping) the gate values in the simulated circuit, which is crucial for analyzing information leakage through alternative thinkers’ CoTs.

Hardness Results: Passive Data Collection

End-to-End Supervision: The first major result establishes that, under cryptographic assumptions (hardness of learning depth-2 threshold circuits, hardness of $GapSVP/SIVP$, or the existence of local PRGs with polynomial stretch), learning the end-to-end task from only final answer supervision is intractable for moderate $T$—already for $T = \omega(\log d \log \log d)$ under worst-case lattice assumptions, or even for constant $T$ under strong PRG assumptions. The improved simulation means the intractability holds for substantially shorter chains than previously shown.

Two or Few Thinkers, Known Identities: The most striking and rigorous hardness result is that even with full CoT traces, if they are sourced from just two thinkers, learning can remain as hard as in the purely end-to-end setting—provided the assignment of thinkers is instance-dependent and the identity selector is computationally hard to learn. This hardness is established by constructing circuit representations with “leakage” on complementary regions of the input space: one thinker reveals only on positive-region examples, the other only on negative-region examples. Through cryptographic reductions, e.g., representing the same Regev-based cryptosystem decryption as both an intersection and union of polynomial threshold gates (see Figure 2), the authors show that learning from such data would break cryptographic primitives.

Figure 2: (Left) The construction of Klivans and Sherstov [klivans2009cryptographic] encodes a decryption function via intersections of degree-2 PTFs; (Right) This work’s corresponding construction via unions of degree-2 PTFs.

Few Thinkers, Unknown Identities, Uniform Sampling: When identities are unobserved and chosen uniformly at random, hardness persists for $\kappa = \omega(1)$ thinkers (again under PRG assumptions). By exploiting the structure of hard-to-learn DNF classes with single-term satisfaction and carefully permuting assignments, the analysis demonstrates that even with compositional statistical redundancy across thinkers, support identification remains intractable unless $s$0 is allowed to scale super-polynomially.

Algorithmic Results: Active and Adaptive Protocols

The paper demonstrates a sharp algorithmic-statistical gap: when the learner is allowed active control—i.e., can select which input/thinking pairs to query for CoTs (“active and adaptive selection”)—the landscape changes dramatically. They present a boosting-based algorithm (AdaBoost over weak learners obtained by running a consistent-oracle on small CoT batches from different thinkers), showing that:

• The number of CoT queries per thinker required is independent of target accuracy $s$1,
• The total number of thinkers grows only logarithmically in $s$2,
• The time complexity is polynomial in problem parameters as long as the base class is tractable.

This conclusion crucially depends on the protocol: in passive or non-adaptive settings (such as batches per thinker determined a priori), the total sample complexity must scale as nearly $s$3 to achieve $s$4 error—exhibiting a significant computational-statistical gap versus the optimal statistical rates of $s$5 for E2E learning.

Numerical and Empirical Results

Through experiment simulations on parity learning (with up to $s$6 CoT generators per target, for support of size $s$7), the authors empirically validate:

• Transformers trained via next-token prediction over mixed-thinker CoTs eventually achieve perfect parity recovery, but the required sample complexity (number of training examples to reach $s$8 accuracy) increases nearly linearly in $s$9 for the mixed-thinker case, while staying almost flat for single-thinker training.

  Figure 3: Test accuracy of a transformer trained on sequences encoding noiseless parities ($n$0, $n$1) versus training iterations. Left: single-thinker; right: multiple-thinker supervision.

  

  Figure 4: Sample complexity—number of examples needed to reach $n$2 accuracy—for single versus multiple thinker training as $n$3 varies, highlighting the computational cost of aggregating CoTs across thinkers.

This demonstrates, both in theory and simulation, a real computational cost to aggregating CoTs from multiple distinct strategies rather than a single source, but also the possibility of practical recovery given sufficient data and the right training protocol.

Implications and Future Directions

This work rigorously demarcates the boundary where the benefit of CoT supervision collapses under practical imperfections of supervision. Collection procedures matter: who generates CoTs, which examples they annotate, whether their identities are known, and whether the learner can actively select queries, all profoundly affect computational tractability.

Implications:

• Curating CoT datasets for reasoning models must consider not just covering skills or solution paths, but also whether the supervision is sufficiently concentrated to allow for efficient learning.
• In settings with many disparate thinkers, active curation protocols or adaptive querying strategies can mitigate intractability.
• Formal separation results motivate further investigation into statistical vs. computational sample complexity tradeoffs, and the precise kind of oracle or aggregation that enables tractable learning.

Future research could probe:

• Tightening lower bounds for smaller numbers of thinkers under weaker assumptions,
• Extending tractable algorithmic results to less adaptive (e.g., only active, not adaptive) settings,
• Characterizing precise tradeoff curves between adaptivity, sample complexity, and computational effort,
• Developing structurally faithful objectives or model classes for CoT faithfulness rather than just end-to-end accuracy,
• Extending the analysis to partial or prefix-only CoTs, emulating common data augmentation practices.

Conclusion

"Learning to Think from Multiple Thinkers" (2604.24737) provides a comprehensive theoretical framework for understanding the computational complexity of learning with CoT supervision in the presence of multiple different, but correct, thinkers. The paper delineates the precise assumptions and protocols under which the power of CoT supervision collapses, shifts, or remains robust, and opens concrete directions for the design of both practical data curation strategies and theoretically grounded CoT learning protocols in LLM and autoregressive systems.