SLA: Scaling Spatial-Language-Action Model via Continual Pre-training

SLA: Scaling Spatial-Language-Action Model via Continual Pre-training

Robots can recognize an object and understand an instruction yet still fail at the physical details that make manipulation reliable: where an object is, how far away it is, and how its position changes as the robot moves. The authors of SLA: Scaling Spatial-Language-Action Model via Continual Pre-training argue that many vision-language-action (VLA) models inherit strong 2D visual and linguistic representations without developing spatial understanding as a general foundation capability.

Their proposed solution, SLA, adds spatial supervision during continual pre-training rather than waiting until task-specific fine-tuning. The reported results suggest that this timing matters: spatial alignment during both stages consistently outperformed alignment applied in only one stage in the paper’s controlled experiments.

The problem with adding 3D at the end

Existing approaches can provide a robot with depth maps, point clouds, or other geometric inputs during adaptation. But this creates practical complications. Depth can be noisy or incomplete, cameras differ in placement and calibration, and many useful robot datasets contain RGB observations without depth. Requiring a new 3D input pathway at deployment can also change the architecture of a pretrained VLA model.

SLA takes a different route. It starts from the pretrained π0.5 policy and trains its internal visual representations to reflect spatial information. The policy continues to use its original action-generation architecture, while the extra geometric machinery is confined to training.

How SLA works

During training, RGB observations are passed to a frozen 3D perception model, OmniVGGT. When available, the teacher also uses depth and camera calibration. Its spatial feature maps become targets for intermediate visual embeddings inside the VLA. SLA then optimizes two objectives: the original action-prediction loss and a cosine-similarity alignment loss between the policy’s visual tokens and the teacher’s spatial representations.

The method includes two mechanisms intended to make this supervision useful across varied robot data. First, feature-dependent gates combine visual embeddings from multiple VLA layers. The authors say a fixed layer is too restrictive because different tasks and embodiments may require different balances of low-level visual detail and higher-level context. Second, probabilistic state dropout occasionally removes the robot’s state tokens during training. This is designed to reduce shortcut learning in which the policy memorizes proprioceptive states associated with demonstration trajectories instead of responding to what it sees.

The authors also curate and normalize six heterogeneous datasets. They report approximately 1.7 million episodes and 14,330 hours before their selection and filtering process. Robot actions are expressed through end-effector poses and a shared coordinate convention, allowing data from different embodiments to contribute to a common control representation. The paper reports that rotation vectors, with relative rotations computed through SO(3) composition, worked better than the other tested rotation representations in one controlled comparison.

What the evaluations show

Across the reported benchmarks, SLA improves robustness and adaptation efficiency. On LIBERO-Plus, it reaches an overall success rate of 87.4% under camera, robot, language, lighting, background, observation-noise, and object-layout perturbations. This is 3.0 percentage points above the π0.5 baseline and 2.3 points above the authors’ post-training-only Spatial Forcing comparison.

On RoboTwin 2.0, SLA achieves 72.3% in the Clean2Random setting and 79.2% averaged across Clean2Clean and Clean2Random. The Clean2Random result is especially relevant because training uses clean demonstrations while evaluation introduces domain variation. The authors report that adaptive layer selection outperformed every tested fixed-layer alternative, while a 25% state-dropout probability was the strongest setting in their ablation.

On RoboCasa-Panda, SLA reaches 68.4% using 50 downstream demonstrations per task. The paper compares this with methods using between 300 and 3,000 demonstrations per task, although those comparisons involve different models and training procedures and should not be interpreted as a controlled data-budget experiment for every baseline.

Results on physical robots are also encouraging. On RoboDojo-Real, one policy adapted across three embodiments achieves a 33.9% overall complete-task success rate, compared with 12.8% for π0.5. In eight additional deployment conditions, the authors report an average success rate of 83.8%, 24.0 percentage points above π0.5. The tests include bimanual handover, objects moved by a person, a moving table, an unseen instructed object, and few-shot adaptation to another robot.

Why continual pre-training matters

The most direct evidence comes from the RoboTwin data-scaling ablation. With spatial alignment enabled during both continual pre-training and post-training, Clean2Random success rises from 49.2% with 5% of the continual-pretraining data to 72.3% with the full set. At full data, alignment during continual pre-training alone reaches 63.2%, while alignment only during post-training reaches 57.1%. The combined approach performs best at every tested data proportion.

This supports the authors’ central claim: spatial understanding is more useful when it shapes the reusable policy representation before downstream specialization. It also suggests that simply adding more robot demonstrations is not equivalent to giving the model an appropriate geometric learning signal.

Limitations and practical implications

The evidence is substantial but not conclusive for every deployment setting. The experiments rely on a particular pretrained policy, a selected set of datasets, and benchmark-specific post-training. The authors’ strongest claims therefore demonstrate improved results within these protocols, not universal spatial reasoning across all robots or environments. Performance also remains uneven: the per-task results show substantial drops under randomized scenes, long-horizon tasks, and tasks absent from post-training demonstrations.

SLA still requires considerable training resources. The paper reports about 110 hours on 48 NVIDIA H100 GPUs for continual pre-training. Although the 3D teacher and alignment modules are discarded at inference, preparing heterogeneous data, handling calibration when available, and maintaining consistent action conventions add engineering complexity.

For practitioners, the main lesson is architectural restraint. A team with a capable VLA model may be able to improve spatial generalization without adding depth or a 3D perception model to the deployed policy. The practical investment shifts toward training data curation, geometric supervision, and carefully designed ablations. The authors’ results make a credible case that spatial competence should be treated as foundation-model knowledge, not merely as a final fine-tuning feature.


Sources

AI-assisted research summary. Check the original paper for complete methods, results, and limitations.

Avatar photo

Tech News

Jeff is a passionate blog writer who shares clear, practical insights on technology, digital trends and AI industries. With a focus on simplicity and real-world experience, his writing helps readers understand complex topics in an accessible way. Through his blog, Jeff aims to inform, educate, and inspire curiosity, always valuing clarity, reliability, and continuous learning.

↑ ↓ navigate Enter open Esc close