WorldFoundry: Towards Unified Infrastructure for World Intelligence
Generative video and 3D systems can produce remarkably convincing scenes, but visual realism is not the same as understanding or simulating a world. A clip may look physically plausible while lacking persistent objects, consistent state changes, or any ability to respond coherently when an agent acts. The authors of WorldFoundry: Towards Unified Infrastructure for World Intelligence argue that this gap is partly an infrastructure problem: models, runtimes, artifacts, and benchmarks are fragmented, making comparisons difficult to reproduce and claims difficult to audit.
What counts as a world model?
The paper proposes an operational definition rather than one tied to a particular architecture. A generative world model should perform three connected functions: infer a latent state from observation history, predict how that state changes under natural dynamics and interventions, and generate action-conditioned responses. New observations must then update the state.
This feedback loop is the paper’s key boundary. A foundational generative model can synthesize or transform observations without maintaining an evolving environmental representation. By contrast, a generative world model must preserve the effects of interventions and use later observations to revise its predictions. The authors emphasize that persistence is demonstrated behavior, not a label attached to a model.
The framework organizes capabilities into perceptual grounding, persistent state modeling, dynamics-aware prediction, and closed-loop interaction. Controllable transformation—editing an object or scene while preserving unrelated content—is treated as an important but orthogonal capability. This progression gives researchers a way to describe systems by capability profiles instead of forcing every model into a single category.
WorldFoundry’s infrastructure approach
WorldFoundry is an open-source system intended to make heterogeneous research workflows more reproducible. It covers video generation, 3D and 4D representations, interactive worlds, and embodied action systems. Rather than replacing each model’s native semantics, the framework places modality-specific adapters around a shared operational lifecycle.
- Discovery and preparation: manifests describe models, tasks, assets, runtimes, dependencies, and known blockers.
- Unified execution: requests preserve native inputs, controls, parameters, and output formats while passing through common orchestration.
- Durable artifacts: outputs such as videos, geometry, trajectories, and state traces receive typed references and provenance records.
- Evaluation and evidence: benchmark runs record coverage, failures, configurations, metrics, and the scope of any resulting claim.
- Interactive access: the project provides terminal, command-line, browser, Python, and programmatic interfaces that share an execution path.
A particularly useful design choice is the separation between execution and evaluation. Once outputs are materialized, they can be inspected, compared, or rescored without reloading the original model. The scorecard also distinguishes diagnostic results, benchmark-comparable results, and exact reproductions. In practical terms, a successful run is not automatically treated as proof of benchmark validity.
What the authors demonstrated
The current catalog contains 270 model entries, 74 benchmarks, and 62 registered metric entries. The paper says that model runners were validated through documented end-to-end workflows, but it also stresses that readiness is route- and configuration-specific. Catalog inclusion, runnable integration, and validated performance remain separate claims.
The empirical study evaluates the infrastructure rather than ranking models across unrelated tasks. The authors report tests of native input and output handling, consistent behavior across the CLI, Python API, and web interface, persistent artifact storage, and traceable evaluation evidence. The experiments used 128 NVIDIA A100 and H100 GPUs with 80 GB of memory each, so the reported operational evidence is tied primarily to that hardware environment.
Across video generation, the authors observed that systems can preserve broad composition and produce smooth motion while still failing on identity, occlusion, fine detail, or contact relationships. For interactive worlds, responsiveness and scene recall are not sufficient evidence of persistent state: revisiting a familiar view does not prove that an opened door stays open or that a moved object remains moved.
The 3D and 4D analysis similarly separates agreement with observed evidence from plausible completion and temporal persistence. Rendered views may look convincing without establishing accurate geometry in unseen regions. In embodied action, the paper presents a closed-loop LIBERO-10 example using OpenVLA: one single-object pick-and-place task completed within 179 steps, while two multi-stage tasks reached the 520-step limit. This is a demonstration of the evaluation workflow, not a broad performance claim about the policy.
Limitations and practical implications
WorldFoundry does not eliminate the underlying difficulty of comparing world models. Models require different hardware, checkpoints, simulators, licenses, hosted services, and evaluation assets. The framework currently focuses on inference, interaction, artifact management, and evaluation; general training infrastructure is left for future work. The authors also say that optimization methods and newer hardware have not yet been systematically characterized across model families.
For researchers, the practical lesson is to make every experiment answerable: What was executed? Under which configuration? Which samples succeeded or failed? Were the official data, evaluator, and coverage used? A manifest-and-artifact workflow can make those answers reusable across teams and benchmarks.
For engineers building agents or simulators, the paper suggests a stricter test for world modeling. Do not rely only on visual quality or offline action accuracy. Test whether an intervention changes later predictions, whether the system remembers off-screen consequences, and whether fresh observations correct mistaken beliefs. These are the behaviors that connect generation to useful interaction.
WorldFoundry is therefore best understood as a measurement and integration foundation, not proof that current systems have solved world intelligence. Its main contribution is to make the distinction testable: persistent state, intervention-conditioned dynamics, and feedback must be backed by executable workflows and traceable evidence.
Sources
AI-assisted research summary. Check the original paper for complete methods, results, and limitations.