AI Agents Solve Research-Level Physics Calculations
TheoPhysEval tests whether frontier AI agents can autonomously solve concrete, verifiable theoretical-physics calculations at research level. Using 50 unpublished problems across 11 specialized domains—from quantum field theory to holography—the benchmark evaluates not theory construction or scientific taste, but the calculational stage where a researcher has a well-posed problem and known techniques. The leading system, GPT-6-Astra, achieves 98% accuracy with web access and 92% without it, demonstrating that agents can now execute many specialized physics calculations with expert-level precision at low cost, even though they cannot yet reliably select important questions or direct research strategy.Script
Can an AI agent sit down with an unsolved theoretical-physics calculation—something a specialist would tackle over days or weeks—and produce the right answer on its own, without human hints?
TheoPhysEval tests exactly that capability with 50 unpublished research problems spanning 11 areas, from lattice theories to holography. These are turn-the-crank calculations—novel in their parameter choices, verifiable by a numerical answer, and deliberately constructed so no public solution exists.
GPT-6 Astra solved 49 of 50 problems with web access—98%—and 46 without it, a drop of only 6 percentage points. That tiny gap suggests the model has already memorized vast portions of the theoretical-physics literature, formulas and all.
But there is a hard boundary. Every agent failed the inverse differential-equation task, where the goal was to infer the equation from a plotted trajectory. Agents excel at forward calculations with clear structure; they falter when asked to reverse-engineer an underspecified mechanism.
The paper makes a critical distinction. Solving a specified calculation is not the same as autonomous research. Agents can now execute the third stage—turning the crank—with expert-level accuracy, but they still lack reliable taste in choosing which problems matter or redirecting a project when the initial formulation proves uninteresting.
Frontier agents can now solve a broad range of specialized physics calculations at low cost and high accuracy, a shift that will reshape how graduate researchers spend their time. To learn more about TheoPhysEval and create your own videos exploring the latest research, visit EmergentMind.com.
