Open-HumanScale: An Open Recipe for Scaling Human and Robot Experience for Embodied Foundation Models

Open-HumanScale: An Open Recipe for Scaling Human and Robot Experience for Embodied Foundation Models

Robot learning has a data problem. Teleoperated demonstrations provide direct control signals, but collecting them is expensive and usually covers a limited range of tasks, objects, and environments. Egocentric human video is far more abundant, yet translating human hand motion into reliable robot behavior requires careful curation, action standardization, and adaptation.

The authors present Open-HumanWAM, described in the supplied paper as a fully open recipe for using human and robot experience to train embodied foundation models. Its central claim is that scaling human video can improve robot generalization, particularly when the data expands in diversity as well as duration. The work also investigates whether combining human and robot data can support zero-shot execution and goal-directed behavior.

Building a usable human-experience corpus

The foundation of the study is Open-HumanWAM-50K, a curated corpus containing 50,000 hours of egocentric manipulation video, 35.9 million continuous clips, language descriptions, and hand-motion annotations. The data comes from five publicly available sources. The authors filter recordings for hand visibility, motion stability, active manipulation, and usable visual context. Idle segments and unstable trajectories are removed rather than allowing large quantities of low-quality footage to dominate training.

A major technical step is aligning heterogeneous action annotations. Human and robot actions are mapped into a common bimanual end-effector representation covering position, orientation, and hand or gripper openness for both hands. The authors also align camera coordinates and wrist axes across sources. Their ablations show why this matters: on a four-source, 1,000-hour mixture, alignment raised best reported success from 72.0% to 77.5% in the clean setting, from 43.2% to 52.2% under scene variation, and from 18.0% to 26.4% under broader randomization.

What scales with more human video?

For controlled experiments, the authors pretrain a world-action model on datasets ranging from 100 to 50,000 hours, then adapt each model using robot demonstrations and evaluate it in closed-loop simulation. The clearest gains appear outside the training distribution. With isolated attention, random-condition success increases from 8.4% at 100 hours to 28.7% at 10,000 hours, while the no-human-pretraining baseline reaches only 2.1%. With a stronger inverse-dynamics configuration, scene-out-of-distribution success rises from 49.1% at 100 hours to 73.2% at 50,000 hours, and random-condition success rises from 21.5% to 42.9%.

The authors fit these trends with an empirical log-linear relationship between pretraining hours and success. They report high goodness-of-fit values for the evaluated settings, but this should be read as an observed scaling pattern over the tested range, not proof that performance will grow indefinitely. Their diversity experiment is equally important: increasing task and scene coverage alongside hours produces steeper fitted gains than adding more trajectories from a fixed coverage.

Human pretraining also improves adaptation efficiency. In one comparison, a model pretrained on 5,000 hours exceeded the no-pretraining model’s scene-shift performance using half as many robot demonstrations. The authors further report that human pretraining outperformed matched-duration robot pretraining from 500 to 5,000 hours in their RoboTwin experiments. This supports the practical idea that diverse human footage can provide useful priors before a robot-specific data collection phase.

HumanWAM-Vanilla: strong transfer after adaptation

Using the full 50,000-hour corpus, HumanWAM-Vanilla is pretrained exclusively on egocentric human data and then post-trained on robot demonstrations. It reaches 92.08% on RoboTwin 2.0 Clean2Clean and 68.88% on Clean2Random. On RoboDojo, the authors report a 14.17% mean success rate and a 19.00 mean progress score.

The physical-robot results emphasize robustness rather than just benchmark fit. Across four tasks, mean in-distribution success rises from 57.5% without egocentric pretraining to 87.5% with it. Under changed tablecloth and lighting conditions, mean success rises from 2.5% to 67.5%. These are notable results, although the physical evaluation uses four tasks and ten trials per task, so they do not establish broad reliability across household robotics.

Hybrid scaling and visual goals

The second model, HumanWAM-Goal, combines robot data with egocentric human video and conditions action generation on a visual goal. In simulation, hybrid pretraining reaches 13.94% mean zero-shot success on RoboTwin 2.0 before target-task adaptation. After post-training, it reaches 92.70% on Clean2Clean and 84.60% on Clean2Random, for an 88.65% average reported by the authors.

On a physical robot, adding 45,938 hours of human data to a fixed 14,618-hour robot corpus raises mean zero-shot success across 20 tasks from 20.0% to 30.5%. Task coverage also expands: at least one successful trial occurs on 10 of 20 tasks with robot-only pretraining, 14 tasks with the smaller human addition, and 16 tasks with the larger one. Robot-balanced sampling improves hard-task success from 19.0% to 27.0% at the same total data scale, suggesting that abundant human data should not overwhelm embodiment-specific robot supervision.

Limitations and practical implications

The evidence is promising but bounded. Many headline results require robot post-training, and the strongest leaderboard result uses additional post-training refinements and augmentation. RoboTwin and RoboDojo remain simulated benchmarks with particular task distributions, while the real-world zero-shot study is relatively small. The claimed embodied in-context learning is demonstrated through selected visual-goal rollouts rather than a broad quantitative generalization study. Goal images used in simulation are synthesized and reviewed for visual consistency, but the authors explicitly note that this review does not verify physical feasibility.

For practitioners, the recipe suggests three priorities: curate active manipulation rather than merely maximizing hours; measure and expand task and scene diversity; and preserve a balanced supply of robot trajectories during human-robot co-training. The open corpus, pipelines, checkpoints, inference code, and evaluation suites could make these choices easier to reproduce. The broader lesson is not that human video replaces robot data. Rather, the authors’ results suggest that human experience supplies scalable manipulation priors, while robot experience remains essential for embodiment, control precision, and dependable deployment.


Sources

AI-assisted research summary. Check the original paper for complete methods, results, and limitations.

Avatar photo

Tech News

Jeff is a passionate blog writer who shares clear, practical insights on technology, digital trends and AI industries. With a focus on simplicity and real-world experience, his writing helps readers understand complex topics in an accessible way. Through his blog, Jeff aims to inform, educate, and inspire curiosity, always valuing clarity, reliability, and continuous learning.

↑ ↓ navigate Enter open Esc close