PhysOPSD: Code as Privilege for Physically Plausible Video Generation
Video generators have become much better at producing sharp images, coherent scenes, and convincing camera motion. Their understanding of physics is less reliable. Objects may stop moving, collide incorrectly, deform implausibly, or fail to respond when a prompt changes a relevant physical condition. PhysOPSD, from Harold Haodong Chen and colleagues, explores whether executable simulation code can provide better training supervision for these failures than text descriptions or reference videos.
The problem with teaching physics to video models
The authors frame physical post-training as a difficult supervision problem. Supervised fine-tuning can provide dense examples, but it trains on fixed, offline data that may not match the states a model encounters during generation. Reinforcement learning can optimize on generated rollouts, but physical rewards are often sparse or difficult to define. A model might receive little useful guidance about exactly when a collision, bounce, spill, or deformation became implausible.
PhysOPSD builds on on-policy self-distillation. The model generates a rollout as a student, while an exponential-moving-average copy of itself acts as a teacher. The teacher receives additional privileged information and supplies dense velocity predictions at the states visited by the student. The student, however, is trained to operate with the text-only conditions available at deployment.
The central design question is therefore what information should be given to the teacher. The authors argue that ordinary text is declarative: it can say that a ball bounces, but does not directly encode the governing interaction. A real video contains motion, but also appearance, identity, background, and camera details that may encourage shortcuts. Executable code offers a different representation: a parameterized, editable physical scene that can be run, inspected, and modified.
How code becomes privileged physical context
The accompanying PhysOPSD-Engine converts physical scenarios into executable simulation programs. Its target scope is deliberately limited to dynamics that can be grounded in standard simulators, including rigid-body and contact interactions, elastic or deformable solids, and fluids. The engine checks whether a program executes, produces physically meaningful dynamics rather than hard-coded animation, makes the key event observable, and responds consistently when physical parameters are changed.
From each verified scene, the system creates a grounded prompt and visual drafts. A dense draft retains detailed geometry and interactions. An abstract draft strips away much of the appearance while preserving coarse motion, trajectories, and contact events. This creates a three-level hierarchy: text alone, text plus an abstract draft, and text plus a dense draft.
Training proceeds through two linked ideas. First, dual-level physical privilege distills knowledge in stages. The dense teacher guides an abstract-conditioned student, and the abstract teacher then guides the text-only student. The intermediate representation is intended to reduce the gap between rich simulation information and deployment-time prompts while discouraging the model from simply copying geometry.
Second, controlled physical variation creates paired scenes that differ in one selected parameter, such as friction, mass, stiffness, restitution, or initial velocity. The paired rollouts use shared initial noise, reducing irrelevant randomness. In principle, this makes the change in generated motion more directly attributable to the changed physical condition.
What the experiments report
Chen and colleagues evaluate the method on two video-generation backbones, UniVideo and OmniWeaving. They compare it with the base models, physics-focused alternatives, supervised fine-tuning, and an offline preference-optimization method. The main training set, PhysOPSD-3K, contains 3,000 executable physical scene instances.
On the authors’ reported results, PhysOPSD improves physical scores on both backbones while retaining general video quality. For UniVideo, the VideoPhy-2 overall score increases by 4.2 points over the corresponding base model, while PhyGround improves by 1.6 points. On OmniWeaving, the reported gains are 4.1 points on VideoPhy-2 and 3.4 points on PhyGround. The authors also report a physical commonsense score of 4.61 for PhysOPSD in both model configurations shown in the table.
Data efficiency is a prominent result. The method uses 3,000 samples, compared with 80,000 for the supervised fine-tuning comparison and 21,000 for the offline reinforcement-learning comparison. On the general VBench evaluation, the reported total quality score rises from 83.5 to 84.0 for UniVideo and remains 83.3 for OmniWeaving after PhysOPSD. These results support the authors’ claim that the method can specialize physical behavior without an obvious loss of broad generation quality.
The ablations are also informative. Under the reported 3K budget, executable drafts outperform refined text and real reference videos as privileged inputs. The authors attribute this to code-derived drafts isolating dynamics and enabling controlled perturbations. Their comparisons further report that the dual-level hierarchy performs better than using only abstract or dense drafts, while controlled parameter pairs outperform independent physical scenes. A scaling experiment reports improvement from a 1K budget through 4K, although these observations remain tied to the tested setup.
Limitations and practical implications
The paper does not claim a universal physics solution. Its pipeline excludes phenomena such as turbulent gas diffusion and thermodynamic phase changes because they are difficult to simulate stably and to vary in a causally clean way. Simulation quality is another dependency: incorrect programs, unstable solvers, or poorly chosen parameters could teach the wrong dynamics. The reported evaluations also focus on two backbones and selected physics and video benchmarks, so broader cross-model and real-world validation would strengthen the conclusions.
There is a practical lesson beyond this specific method. For teams improving generative video, executable representations may be useful not because the final system must run a simulator, but because simulation can provide structured training signals. Carefully designed abstractions, paired counterfactuals, and on-policy supervision could help models learn relationships that static captions obscure. At inference time, the authors report that the teacher, simulator, and physical drafts are discarded, leaving a text-to-video or text-and-image-to-video generator without added simulation runtime.
PhysOPSD’s demonstrated contribution is therefore a training strategy: use code to expose controllable physical causes, then distill the resulting guidance into the generator. Whether that strategy transfers to harder, less simulation-groundable phenomena remains an open question, but the reported results make code-based privilege a compelling direction for physically grounded video generation.
Sources
AI-assisted research summary. Check the original paper for complete methods, results, and limitations.