Can Language Models Discover Physical Laws Without Being Told What to Find?
This presentation examines a stringent test of autonomous scientific reasoning: can a large language model notice an anomalous physical event in a simulated world, decide on its own to investigate it, design experiments that rule out competing explanations, and report a physical law supported by evidence? The LongChang benchmark removes the usual scaffolding—no assigned research question, no hints about which objects matter, no specification of what equation to recover—and evaluates the full chain from observation to discovery. The results reveal a stark capability gap: among 12 models tested, only one achieves any successful discoveries, and even that model succeeds in fewer than one case in six.Script
Most benchmarks tell language models exactly which equation to find. But what happens when you remove that instruction entirely and ask a model to explore a physics simulation, notice something strange, and figure out the hidden law on its own?
The LongChang benchmark makes that distinction precise. Traditional tasks specify the target law in advance, turning discovery into equation fitting. Here, the explorer receives only a simple instruction to explore freely and report what it learns. It is not told that physics has been altered, which objects are relevant, or what phenomenon to investigate.
The framework constructs simulated worlds in MuJoCo, each containing one altered physical law. An explorer manipulates objects through pushing, throwing, and waiting, and can even design custom measurement apparatus by submitting physics specifications. Separate agents then evaluate whether the final report correctly identifies the law and whether the exploration record contains experiments sufficient to support that claim. Unresolved questions and corrected errors feed back into an automated scene-evolution loop, creating harder challenges.
Among 12 evaluated models, only GPT 6 Astra achieves any fully successful discoveries, with a rate of 16.7 percent. The other 11 models all score zero. This is not because they fail to interact. Some explore for more than 90 turns and successfully construct apparatus, yet they never complete the chain from anomaly to validated law.
The most revealing failures occur when a model observes something unexpected but never converts it into a research question. In one scene, GPT 6 Astra notices a delayed restart, suspends its original plan, and designs experiments to measure the pause duration and test whether other objects trigger the same effect. Gemini Flash observes an identical event but continues its predetermined exploration, leaving the anomaly unexplained. That difference isolates the transition from noticing to investigating.
LongChang shows that free exploration is far harder than recovering a law when someone tells you what to look for. The bottleneck is not interaction volume or tool access, but the ability to notice what matters, design experiments that discriminate among explanations, and report only what the evidence supports. If you want to explore more research like this or create your own video summaries, visit EmergentMind.com.
