LongChang: Framework for Autonomous LLM Discovery

• LongChang is an automated framework designed to assess whether large language models (LLMs) can autonomously discover altered physical laws through unguided experimentation in virtual environments, particularly worlds simulated using the MuJoCo simulator. The framework evaluates the LLM's ability to observe events, formulate hypotheses, design experiments, and report findings without being given a specific research task or clue.
• It includes 15 scenes with various altered laws, such as red-sphere attraction and oscillating gravity, that require controlled interventions and observations to differentiate between competing hypotheses. The main models evaluated were GPT-5.6-Luna, GPT-6-Astra, Claude-Opus-5, and nine other LLM variants, with GPT-6-Astra achieving the highest discovery rate of 16.7% and supported report rate of 19.0% correct
• The framework is designed to evaluate whether LLMs can autonomously discover and report altered physical laws in virtual environments, addressing independent inquiry

LongChang is an automated framework for evaluating whether LLMs can autonomously discover altered physical laws through unguided experimentation. It places an LLM explorer in simulated MuJoCo rigid-body worlds whose physical rules may differ from ordinary physics, without specifying the research question, relevant variables, anomaly, or target law. The explorer must observe events, identify possible anomalies, formulate hypotheses, manipulate objects, construct experimental apparatus, collect evidence, and produce a report. LongChang evaluates both whether the reported law is correct and whether it is supported by the recorded experiments. In the reported benchmark, GPT-6-Astra achieved a discovery rate of 16.7%, whereas the other 11 evaluated LLMs achieved 0% (Yang et al., 27 Oct 2026).

1. Research objective and conceptual basis

LongChang evaluates a form of scientific autonomy stronger than conventional law-discovery benchmarks. In many existing settings, an agent receives the equation to recover, the variables to investigate, or an explicit anomaly to explain. LongChang removes these supports. The explorer is instructed essentially to “Explore freely and report what you learn,” without a research task or warning that the laws may be altered.

The intended process comprises seven stages:

1. observing an unexpected event;
2. recognizing it as anomalous;
3. formulating a research question;
4. designing interventions that distinguish competing hypotheses;
5. collecting and recording evidence;
6. inferring the altered law, including its targets, conditions, triggers, and parameters; and
7. reporting the result with reproducible support.

The benchmark therefore evaluates “independent inquiry” rather than isolated prediction, equation fitting, or execution of a pre-specified experiment. A successful explorer must decide not only how to answer a question, but which question is worth asking.

The name LongChang refers to Wang Yangming’s introspection at Longchang and the associated idea that knowledge is pursued through active investigation rather than passive reception. Within the benchmark, this principle is operationalized as autonomous interaction with an environment and evidence-based reporting.

The motivating example is an object whose behavior violates ordinary expectations, such as heavier objects falling more slowly. A model that merely describes the observation has not completed the task. It must determine whether mass affects falling speed, construct an appropriate comparison, distinguish that explanation from alternatives, and specify the resulting law and its scope.

2. Simulated physical worlds

Each LongChang world is a virtual “glass room” implemented in the MuJoCo rigid-body simulator. The environment contains manipulable objects such as colored balls, boxes, floor tiles, walls, and apparatus constructed by the explorer.

The ordinary control world follows standard rigid-body behavior:

$\mathbf F=m\mathbf a,\qquad \mathbf p=m\mathbf v,$

where $\mathbf F$ is force, $m$ is mass, $\mathbf a$ is acceleration, $\mathbf p$ is momentum, and $\mathbf v$ is velocity. Gravity is downward and independent of mass; collisions and friction behave ordinarily; and colors and decorative tiles have no physical effect. The altered law is hidden from the explorer but retained by the scene designer and report evaluator.

The base benchmark contains 15 scenes:

Scenes	Condition
Scene 01	Ordinary-physics control
Scenes 02–15	Fourteen scenes with one altered law each

The altered laws involve dependencies on object identity or color, mass, time, velocity, speed thresholds, position, contact events, collision counts, past actions, and delayed consequences. Examples include:

• Red-sphere attraction: red spheres attract one another according to

$\mathbf F=20\frac{\mathbf d}{\max(r,0.5)^3},$

where $\mathbf d$ points toward the other sphere and $r=|\mathbf d|$.

• Mass-dependent falling: in Scene 03, a free body’s downward acceleration is

$g_0\sqrt{\frac{m_0}{m}},$

with $\mathbf F$0, so heavier bodies fall more slowly.

• Oscillating gravity: Scene 04 uses

$\mathbf F$1

• High-speed braking: Scene 05 applies

$\mathbf F$2

where $\mathbf F$3.

• Tile-dependent rebound: impacts on red tiles have restitution $\mathbf F$4, while impacts on blue tiles have restitution $\mathbf F$5.
• Contact color exchange: in Scene 12, free single-sphere bodies swap colors once at the beginning of each separate contact.
• Third-contact jump: every third red-blue contact in Scene 13 gives both objects an upward velocity of $\mathbf F$6.
• Delayed velocity copying: in Scene 14, each launch of red schedules green to take red’s three-dimensional linear launch velocity 12 seconds later.
• Temporarily paused force: in Scene 15, a red ball experiences an additional northward force

$\mathbf F$7

Every nonzero push or throw of any movable object restarts a ten-second pause during which the force is suppressed.

These laws are designed to require controlled interventions rather than single observations. For example, two successive red-blue collisions may be consistent with either color exchange or independent color flipping. Testing two red balls can distinguish the alternatives: under exchange, both remain red; under independent flipping, both become blue.

3. Explorer interface and experimental agency

The evaluated LLM acts as an explorer. It receives structured observations containing object positions, motion and trajectories, object states, instrument readings, elapsed counts, and execution feedback. Distances are expressed in “bricks,” and time is expressed in counts; one count corresponds to one simulation second, while the underlying MuJoCo integration time step is $\mathbf F$8 seconds.

Available actions include:

• PUSH, which adds a horizontal impulse;
• THROW, which applies an impulse with heading and elevation;
• WAIT, which advances the scene;
• GRASP, MOVE, and RELEASE, which manipulate held objects;
• MARK, which creates a nonphysical reference point;
• BUILD, which constructs an apparatus;
• PLACE, which puts an object at a tool port;
• ADJUST, which moves a hinge or slide; and
• DONE, which terminates exploration and requests a report.

For example,

$m$8

adds a horizontal impulse, while

$m$9

adds a directed impulse.

The explorer can construct physical tools through BUILD commands containing MuJoCo XML/MJCF. Tools can include bodies, geometry, mass or density, friction, hinges, slides, springs, tendons, sites, and mechanical connections. A tool is therefore not an abstract sensor selected from a fixed catalog; it is an experimental intervention that must itself be designed and validated.

One example involves a weighing platform. After a ball rolls off the initial platform, the explorer adds retaining walls and increases the platform’s vertical travel to obtain stable measurements. Another apparatus is a pendulum whose hinge angle can indicate whether local gravity is tilted.

The explorer’s note precedes each action and records its interpretation or rationale. This requirement prevents retrospective rationalization: a final report cannot claim that an action was designed to test a hypothesis if that hypothesis was absent from the prior exploration record.

4. Multi-agent benchmark architecture

LongChang separates the explorer from supporting LLM roles responsible for scene generation, review, tool validation, and report assessment.

Scene designer

The scene designer creates physical worlds by specifying objects, geometry, physical programs, ordinary and altered laws, target objects, conditions, triggers, expected effects, and parameters. In the evolving benchmark, it can generate a new law, refine an existing scene by adding conditions or dependencies, or compose multiple laws in one environment.

Scene modifications must cite exploration evidence. Automated tests execute the resulting scene, including interactions with constructed apparatus. Failed compilation, failed tests, or reviewer rejection causes revision requests.

Scene reviewer

The scene reviewer verifies whether a generated or modified scene implements the intended law. It examines the physical program, automated simulation tests, law conditions and parameters, and behavior under relevant interventions. Only accepted scenes enter later evaluations.

Tool-design reviewer

The tool-design reviewer checks whether proposed MJCF apparatus compiles, whether its geometry and joints are physically valid, whether it is compatible with the environment, and whether it can perform its intended role. Rejected designs generate feedback that permits the explorer to revise and resubmit the apparatus.

Report evaluator

GPT-6.1-Sol evaluates reports against the hidden ground-truth law and the complete exploration record. It checks correctness and support separately, using the original actions, observations, execution feedback, and cited evidence.

The evaluation process rejects invented observations, unsupported reasoning, and explanations introduced only after the experiment. The explorer’s final report must contain an experiments array and a conclusions array. Each experiment identifies the turn, actual action, setup conditions, outcome or execution feedback, and observed result. Each conclusion contains the claim, preconditions, cited evidence turns, and reasoning.

The complete cycle is:

$\mathbf F$9

In the main benchmark, only the explorer is the evaluated model. The supporting agents assist with benchmark construction and evaluation. In the evolving benchmark, exploration records are fed back into scene design to create more difficult experiments without human intervention.

5. Metrics and evaluation criteria

The main protocol uses the same 15 scenes for all models, fixed initial states, a maximum of 100 turns per run, free exploration, and no research task or altered-law clue. GPT-6-Astra is also used as the apparatus reviewer and scene-generation/review model.

The 12 evaluated explorers are GPT-5.6-Luna, GPT-6-Astra, Claude-Opus-5, Gemini-3.7-Flash, GLM-5.3, GLM-5.3-Flash, Kimi-K3, MiMo-V2.6-Flash, MiMo-V2.6-Pro, DeepSeek-Flash, Qwen3-Max, and Grok-4.7.

A discovery requires both law correctness and experimental support:

$m$0

where $m$1 denotes correctness and $m$2 denotes support.

Correctness

A report is correct when it identifies the altered law, including applicable target objects, object identities or categories, conditions, triggers, constants or numerical parameters identifiable in the reported units, and the law’s scope. An alternative mechanism can count as correct if it produces the same observable predictions under all permitted interventions.

Observing several matching outcomes is insufficient if the report omits a crucial trigger or applicability condition. A report may therefore describe a genuine effect while failing to identify the complete law.

Support

Support requires that the experimental record justify the claims at the certainty level used in the report. The evaluator checks whether the actions occurred, whether the observations match the report, whether comparisons control relevant variables, whether cited turns contain the required evidence, whether alternative explanations remain unresolved, and whether unsupported assumptions are presented as facts.

A correct guess can pass correctness while failing support. Conversely, a careful report can document a real anomaly without identifying the complete law.

Control rejection

The ordinary-physics control measures overinterpretation. Control reports are rejected when they assert an anomaly, unsupported law, or incorrect explanation of ordinary behavior. Across models, 36.1% of control reports were rejected.

Additional exploration statistics include:

• $m$3: mean exploration length in turns;
• $m$4: percentage of runs ending before the turn limit;
• $m$5: percentage of turns with failed actions;
• $m$6: number of explorations with at least one successful tool build; and
• $m$7: percentage of failed build attempts.

6. Results, failure modes, and significance

The central result is that only GPT-6-Astra achieved a nonzero discovery rate. It reached 19.0% correctness, 97.6% support, and 16.7% discovery, while the other 11 evaluated models achieved 0% discovery.

Model	Discovery	Correctness	Support	Control rejection
GPT-5.6-Luna	0.0%	0.0%	38.1%	0
GPT-6-Astra	16.7%	19.0%	97.6%	0
Claude-Opus-5	0.0%	16.7%	0.0%	0
Gemini-3.7-Flash	0.0%	0.0%	19.0%	0
GLM-5.3	0.0%	4.8%	0.0%	3
Kimi-K3	0.0%	2.4%	0.0%	3
MiMo-V2.6-Pro	0.0%	7.1%	0.0%	2
DeepSeek-Flash	0.0%	0.0%	0.0%	2
Qwen3-Max	0.0%	0.0%	9.5%	0
Grok-4.7	0.0%	0.0%	2.4%	1

GPT-6-Astra’s high support rate but low correctness rate separates evidence-based experimentation from complete law identification. It usually produced reports grounded in actual experiments, but rarely recovered every target, condition, trigger, delay, and parameter. Claude-Opus-5 showed the opposite pattern: it sometimes stated the correct law but did not support the claim with adequate experiments.

The principal failure modes are:

• Ignoring anomalies: a model observes an unexpected event but continues its original plan rather than investigating it.
• Premature stopping: models terminate exploration while important conditions remain unresolved.
• Incomplete law identification: a model discovers an effect but omits spatial conditions, temporal triggers, object scope, thresholds, delays, or applicability.
• Poor controls: a model changes multiple variables simultaneously and attributes the result to the wrong factor.
• Tool-construction failures: apparatuses fail to compile, function incorrectly, or produce unstable measurements.
• Overinterpretation: ordinary physics is reported as an altered law.
• Persistence without inquiry: long interaction sequences do not necessarily produce useful research questions or discoveries.

In Scene 15, GPT-6-Astra observes a red ball moving, pushes it, sees its motion stop, measures the delay before motion resumes, and tests whether moving another object resets the delay. It eventually identifies the northward force, the launch-triggered pause, and the ten-second duration. Gemini observes a similar restart but does not abandon its existing plan to investigate the event. This contrast distinguishes anomaly perception from anomaly-driven inquiry.

Ablations show that guidance improves discovery. When GPT-6-Astra receives a law-discovery task, a warning that laws may be altered, and a scene-specific clue, discovery increases from 16.7% to 28.6%, while correctness increases from 19.0% to 31.0%; support remains 97.6%. Increasing the turn limit from 100 to 500 does not improve discovery: discovery falls from 16.7% to 14.3%. Access to apparatus is also consequential: discovery is 16.7% with apparatus access and 4.8% without it, although this comparison does not establish that every successful discovery used a tool.

Scene evolution creates supported but incomplete findings. GPT-6-Astra achieves discovery rates of 33.3% on Scene 15 and 66.7% on Scene 14, but 0.0% on the generated, refined, and composed variants. The evolved scenes target previously unresolved dependencies, such as pauses, spatial conditions, and independently timed rules. This indicates that the benchmark can convert earlier exploration errors into more difficult experimental challenges.

LongChang’s conclusions are bounded by its simulated setting. Its laws are programmatically known to the evaluator, its environments are MuJoCo worlds, and its altered rules are deliberately designed to be discoverable through manipulation. Report evaluation depends on an automated judge, and the benchmark does not yet address genuinely unknown scientific claims for which no hidden ground truth exists. The results therefore do not establish general autonomous scientific discovery. They show instead that, under controlled conditions, an LLM can occasionally complete the cycle from anomaly recognition to supported law identification, while broad language competence, persistence, and tool use alone do not reliably produce independent inquiry.