WorldDraft: If Code Builds the World, Can Video Models Just Render It? asks whether video generators can perform a narrower, more reliable job: take a programmatically simulated world and turn it into a realistic video without changing what happens inside that world.
The idea is to divide the work. A coding agent and a simulator such as Blender or MuJoCo define objects, trajectories, interactions, and camera motion. A video model then acts as a renderer, adding the requested colors, materials, lighting, and realism. In principle, this could avoid asking one model to infer both physics and appearance from a short text prompt. In practice, the authors find that current models still struggle to keep those responsibilities separate.
The benchmark tests two kinds of correctness
The authors identify two requirements for a useful generative renderer. World preservation means retaining entity identities, spatial layout, event timing, object motion, physical interactions, and camera movement from the draft. Visual realization means applying the target appearance to the correct entities and keeping those attributes attached through motion, occlusion, deformation, and viewpoint changes.
These goals can conflict. A model may preserve the rough motion while ignoring the requested appearance, or follow the text prompt by inventing a different scene. Standard video-quality metrics do not directly expose that trade-off, so WorldDraft evaluates the two dimensions separately, along with visual quality and appearance persistence.
A controlled 3 × 3 evaluation grid
WorldDraft contains 432 paired test cases built from 48 executable 4D world families. The worlds cover six types of difficulty:
- Multi-entity orchestration
- Multi-stage dynamics
- Occlusion and reappearance
- Camera motion
- Non-rigid deformation
- Hybrid physical interactions
Each world is rendered at three draft-granularity levels. An abstract draft provides primitive shapes and basic motion; a simulation draft supplies more conventional meshes and lighting; and a previs draft adds denser geometry and richer visual context. The same dynamics are also paired with three appearance conditions: neutral source materials, source materials aligned with the target prompt, and deliberately conflicting source materials.
This factorial design lets the authors ask whether more detailed drafts help, and whether a model can preserve structure while overwriting misleading surface cues. The evaluation combines tracking, object counts, bounding-box overlap, event timing, camera estimates, attribute-level visual question answering, and correctness-gated temporal persistence.
What the model comparisons revealed
The authors evaluate 12 video models and report a clear tension between structure and appearance. Models that preserve the simulated world better are not necessarily the ones that best realize the target prompt. For example, SolarRenderer records the strongest reported preservation score among the highlighted examples, while Wan 3.0 achieves the highest reported realization score but has the weakest world-preservation result in the table.
The paper also reports that richer drafts reliably improve structural preservation but often reduce appearance fidelity. Under neutral appearance conditions, moving from an abstract draft to a detailed previs draft produces positive structural gains across the open-weight models. However, appearance and binding gains are negative for nine of the ten open-weight models listed in that comparison. The authors attribute this pattern to dense synthetic shading and other visual details becoming strong priors that compete with the text instruction.
Conflicting source cues expose another weakness. When a draft shows one color or material but the target prompt requests another, models frequently retain the source appearance. The reported source-leakage rates reach 64.17% for Wan2.1-VACE and 52.24% for UniVideo; even several larger or commercial systems show leakage of roughly 28% to 30% in the reported conditions. These results suggest that many video models treat draft pixels as commitments rather than editable evidence.
The qualitative failures are consequential, not merely cosmetic. The paper describes models changing camera paths, rearranging shelving, dropping entities, scrambling colors in multi-object layouts, and retaining conflicting textures through motion. A visually polished output can therefore be physically wrong.
WorldDraft-Primer: steering from the first frame
To address these failures, the authors introduce WorldDraft-Primer, a test-time visual prompting strategy that changes only the first draft frame. It adds sparse object-emphasis markers to help the renderer locate important entities, plus localized color, texture, shading, and environmental cues that represent the requested appearance. The remaining draft frames are left untouched, preserving the original motion sequence.
In the reported tests, the method improves both objectives for OmniWeaving and MiniMax-H3. For OmniWeaving, the authors report a 2.99-point preservation gain, a 4.56-point realization gain, and a 2.51-point overall improvement, while source leakage falls by 3.42 percentage points. For MiniMax-H3, realization rises to 80.21 and the overall score reaches 65.56. Text-only prompt engineering produces much smaller improvements in the same comparison.
The authors’ ablation supports the intended division of labor: removing object emphasis damages spatial metrics, while removing attribute cueing reduces appearance realization and persistence. They also caution that the method depends on the underlying renderer. Weaker models may echo the visual markers, and dense previs drafts can still cause artificial shading or other appearance interference.
Limitations and practical implications
WorldDraft is a useful diagnostic, but its evidence has boundaries. The benchmark uses executable synthetic worlds rather than unrestricted real-world scenes, so success on it would not guarantee reliable rendering in every production environment. The evaluation also relies substantially on automated visual question answering and model-based perception, although the authors report agreement with five human judges on sampled cases. Only 12 models are tested, and the primer experiments cover representative models rather than the entire benchmarked set.
For developers building code-driven video pipelines, the practical lesson is to measure structure and appearance separately. A high aesthetic score is not evidence that trajectories, identities, camera paths, or event ordering survived. Neutral and conflicting source conditions are especially valuable stress tests. The paper also suggests that richer intermediate renders should not be assumed to help: additional detail may anchor geometry while making target appearance harder to control.
WorldDraft-Primer is best viewed as a promising lightweight intervention, not a complete solution. The authors demonstrate that carefully placed visual cues can improve frozen renderers without retraining, but their findings still point to a deeper need: video models must learn which parts of a draft define the world and which parts are merely surface information that should be replaced.
Sources
AI-assisted research summary. Check the original paper for complete methods, results, and limitations.