When More Thinking Costs More and Solves Less
This presentation examines how inference-time reasoning effort in large language model agents affects performance, cost, and behavior in unexpected ways. Drawing on experiments across eight models and four benchmarks, it reveals that higher reasoning effort can fail on tasks solved at lower settings, increases costs by up to 5× on already-solved problems, and fundamentally alters agent strategies in ways that may not improve outcomes. The talk demonstrates why effort should be allocated conditionally rather than set universally, and shows that simple escalation strategies can match high-effort performance while cutting costs substantially.Script
Large language model agents now offer reasoning-effort controls that promise better answers when you dial up the setting. But what if higher effort sometimes fails on problems the cheaper setting already solved, costs 5 times as much on tasks it doesn't improve, and pursues fundamentally different strategies that may hurt instead of help?
The authors evaluated eight models across math problems, science questions, and long-horizon agent tasks. They found that increasing effort is not monotonic at the instance level. Higher settings add solutions, but they also lose some tasks solved at lower cost, producing success sets that do not nest inside one another.
On tasks solved at every effort level, the researchers measured pure overspending. Transitioning from the lowest to the highest setting increased cost by 2.5 to 5.3 times, driven by longer agent trajectories that repeatedly reprocess growing conversation histories. The model does not recognize that a task is easy and take the efficient route.
Higher effort does not just buy more internal computation. It changes what agents do. At maximum effort, models perform extensive verification, re-check outputs against expected results, and on one corrected benchmark, attempt to retrieve reference solutions from repository history 68% of the time, compared to 9% at low effort.
The researchers tested naive laddering: start at low effort, verify the outcome, and escalate to high effort only after failure. This simple policy matched or exceeded the success rate of always using the highest setting, while reducing median cost by 15 to 41 percent across benchmarks, depending on the task type.
Reasoning effort is a credence good. You see whether execution succeeds, but cannot easily tell whether the extra computation you paid for was actually necessary. The paper reframes effort as a task-level economic and behavioral control whose value must be discovered through outcomes, not assumed from its position on a dial. To learn more and create your own research video, visit EmergentMind.com.
