Can LLMs Uncover Hidden Physical Laws through Free Exploration?
Scientific discovery does not always begin with a clearly defined question. Sometimes it starts with a surprising observation: an object falls at an unexpected rate, a measurement changes after an unexplained intervention, or two competing explanations fit the same initial evidence. The researchers behind Can LLMs Uncover Hidden Physical Laws through Free Exploration? ask whether large language models can make that transition from anomaly to research question without being told what to investigate.
To study the problem, Yang and colleagues introduce LongChang, a framework for testing autonomous physical inquiry in simulated worlds. The central challenge is deliberately open-ended. An explorer receives no assigned law-discovery task and no hint about which physical rule has been changed. It must observe a virtual environment, decide whether something is unusual, formulate a question, design interventions, and produce a report supported by its recorded experiments.
A benchmark for noticing anomalies
LongChang places an LLM in a simulated “glass room” built with the MuJoCo physics engine. The base evaluation contains one ordinary-physics control scene and 14 scenes in which a single law is altered. The changes include effects such as mass-dependent falling, oscillating gravity, unusual collision behavior, velocity copying, and a force that temporarily pauses after an object is launched.
The explorer can push, throw, grasp, move, release, and wait. It can also build apparatus when a measurement requires more control. For example, the paper describes a pendulum for testing the direction of gravity and a redesigned weighing platform whose retaining walls prevent an object from drifting away. The tool is not supplied as part of a fixed experiment: the model decides whether it needs one and specifies its construction.
LongChang separates exploration from evaluation. Other LLM agents design and validate scenes, review proposed apparatus, and judge final reports against the hidden laws and the complete exploration record. A discovery must satisfy two conditions: the reported law must be correct, and the claimed conclusion must be supported by actual experiments. This distinction matters because a model can guess a rule correctly without demonstrating it.
Only one evaluated model discovered a law
The authors evaluate 12 LLMs on the same 15 scenes, with a limit of up to 100 turns per exploration. Their headline result is stark: GPT-6-Astra achieves a 16.7% discovery rate, while the other 11 models score 0% under the paper’s combined correctness-and-support measure.
Even the strongest result is far from complete scientific discovery. GPT-6-Astra’s evidence-support rate is 97.6%, but its law-correctness rate is only 19.0%. In other words, the model often made observations that were genuinely supported while failing to identify the full scope, trigger, or conditions of the altered rule. The paper gives an example in which a report supports attraction between tested red balls but does not establish how the effect depends on distance or whether it generalizes to other red spheres.
The contrast with Claude-Opus-5 is revealing. That model identifies altered laws in 16.7% of runs, but none of those reports passes the evidence-support check. The result illustrates why “the model named the right answer” is not enough for evaluating scientific agents.
Curiosity and experimental discipline are separate abilities
The exploration records show that models can miss the most important moment in an investigation: the unexpected observation. In one case study, GPT-6-Astra notices that a ball starts moving unexpectedly, postpones a planned collision test, and investigates the restart delay. It then tests whether pushing another object produces the same pause and identifies the altered rule. Gemini-3.7-Flash observes a comparable anomaly but continues with its original plan, leaving the cause unexplored.
The authors also test whether models can distinguish competing explanations. In a color-changing scene, repeated observations are consistent both with color exchange and with independent color flipping. A test using two red balls produces different predictions for those hypotheses; the unchanged colors support exchange and rule out flipping. This is the kind of discriminating experiment that turns an observation into evidence rather than merely accumulating more examples.
More resources alone do not solve the problem. Raising GPT-6-Astra’s budget from 100 to 500 turns does not improve discovery because the model typically stops after roughly 40 turns. Giving explorers access to apparatus increases discovery from 4.8% to 16.7% in the reported comparison, but the authors caution that this measures access to construction, not the isolated contribution of successful tool use. Action failures and failed build attempts remain common for several models.
Scene evolution exposes unresolved dependencies
LongChang also uses exploration records to generate harder scenes. Separate LLM designers receive unresolved questions and corrected errors, then generate, refine, or compose new physical laws. In the paper’s reported test, GPT-6-Astra achieves discovery rates of 33.3% and 66.7% on two source scenes, but 0% on all three evolved scenes. Evidence support remains high, yet the model fails to recover additional conditions such as position dependence, spatial triggers, or independently timed effects.
This result suggests that partial understanding does not automatically transfer to more complex environments. A model may correctly notice a pause in one setting while missing the interaction between that pause and a second delayed rule.
What the results mean in practice
LongChang is best understood as an evaluation framework, not evidence that LLMs are ready to conduct unsupervised science. The demonstrated results show substantial weaknesses in anomaly detection, question formation, controlled comparison, persistence, and complete law specification. They also show why evaluations should preserve full trajectories and distinguish supported partial findings from correct discoveries.
For developers building scientific or engineering agents, the practical lesson is to reward follow-up on anomalies rather than only completion of a prewritten plan. Systems should track untested assumptions, require discriminating experiments, and audit whether conclusions are grounded in observations. Longer context or larger action budgets may help only when the agent uses them effectively; otherwise, better stopping policies and question-selection strategies may matter more.
The authors emphasize that these experiments occur in simulated worlds with known ground truth. Real scientific questions lack an evaluator that already knows the correct law, and generated experiments or apparatus could be unsafe without independent validation. LongChang therefore demonstrates a difficult capability in a controlled setting, while leaving open the harder question of whether an LLM can make reliable, genuinely novel discoveries beyond a benchmark.
Sources
AI-assisted research summary. Check the original paper for complete methods, results, and limitations.