Why long-horizon game agents are difficult
Game-playing AI has moved well beyond predicting the next button press. Modern 3D games require an agent to interpret raw pixels, navigate changing environments, manage objectives, interact with menus, remember earlier events, and recover from mistakes. A task may span many minutes or hours, while the game continues advancing during inference.
The authors of Gamer: A Generalist Vision-Language-Action Foundation Model for Long-Horizon Real-Time Game Agent argue that these conditions make complex games a useful testbed for generalist vision-language-action systems. Unlike agents that access privileged game state or game-specific APIs, Gamer observes the screen and acts through ordinary keyboard and mouse inputs. That makes its interface closer to how a human player interacts with software.
A three-stage recipe for generalist control
Gamer combines continued pre-training, supervised fine-tuning, and online reinforcement learning. The first stage learns a transferable control prior from gameplay trajectories collected across multiple environments. The researchers normalize different games into a shared keyboard-and-mouse representation and use the game’s control specification to explain what each input does.
Two training adjustments target common problems in gameplay data. Random key remapping discourages the model from memorizing that a particular physical key always has a particular meaning. Repeat-aware loss decay reduces the influence of long runs of identical actions without deleting information about how long an action was held. The authors report that more than 85% of action steps repeat the preceding action, so emphasizing behavioral transitions may help the model learn when to stop, turn, or interact.
The second stage adds task conditioning and memory. Supervised examples cover three levels: local operations, intermediate subgoals, and complete task stages. The model can act directly during routine moments, generate explicit reasoning at decision points, or produce a memory summary before older context is discarded. These summaries preserve information such as completed objectives, failed attempts, important locations, resource states, and remaining plans.
Finally, Temporal-Aware Group Relative Policy Optimization, or TA-GRPO, improves the policy through online interaction. Long rollouts are divided into memory-linked segments, allowing training to continue beyond a single context window. Environment-verifiable outcomes provide rewards, and relative advantages are assigned within groups of rollouts. A temporal weighting scheme gives more emphasis to actions nearer the verified segment outcome, while preserving the average learning signal.
Real-time inference is part of the method
The paper treats latency as more than a deployment detail. If the game changes while an action is being generated, the action may be based on an obsolete observation. Those timing errors can accumulate across a long trajectory and also corrupt reinforcement-learning feedback.
To address this, the authors build Streaming-vLLM, a serving system shared by training and evaluation. It reuses cached context across turns, compacts old history into persistent memory, and supports concurrent game environments. It also uses finite-state-machine-guided speculation for predictable output structure and streams early action blocks before the complete action packet is ready.
In one measured H200 configuration using a Qwen3.5 mixture-of-experts model, the paper reports a cumulative reduction in end-to-end latency from approximately 1.31 seconds to 65 milliseconds. The authors explicitly distinguish this throughput-derived capacity from the game client’s realized control rate, so the figure should not be read as a direct measurement of gameplay frame rate.
What the experiments show
The controlled GameScaling study covers 120 continued-pre-training configurations. Its central finding is that model size and gameplay hours are not enough by themselves: gains depend strongly on whether the training mixture resembles the target environment. Mixed-game data support transfer to related environments, while additional data provide limited sustained improvement on more distant unseen games.
On four public game benchmarks, the reported real-time scores for Gamer’s 8B dense and 35B-A3B mixture-of-experts continued-pre-training checkpoints average 29.5 and 31.5, respectively. The authors report that both outperform the listed general-purpose baselines on the aggregate metric. The mixture-of-experts checkpoint also has a positive average difference between real-time and time-paused evaluation, which the paper connects to faster inference.
The strongest evidence comes from held-out tasks within Honkai: Star Rail. On two side quests, supervised fine-tuning raises overall milestone accuracy from 34.4% and 47.9% after continued pre-training to 72.2% and 69.1%. TA-GRPO then increases those figures to 80.0% and 83.6%. On a seven-type minigame benchmark, the macro-average rises from 15.0% after continued pre-training to 39.2% after supervised fine-tuning and 49.7% after reinforcement learning.
These results support the authors’ claim that task supervision, persistent memory, and online feedback contribute capabilities beyond low-level action imitation. They do not show that Gamer can solve arbitrary games or reliably complete every long-running quest.
Limitations and practical implications
The evaluation focuses on selected environments, with the detailed long-horizon tests concentrated in a familiar game rather than a wholly unseen world. The paper’s own scaling analysis finds that distant unseen games can plateau or worsen as more training data are added. That makes data composition and coverage a central engineering constraint.
The system also depends on substantial infrastructure: the reported reinforcement-learning setup uses parallel Windows game environments and a large H200 inference pool. The authors note that serving costs remain a barrier to wider deployment, and that deeper strategic reasoning may still be a bottleneck even when reactive control is fast.
Practically, Gamer points toward agents for automated gameplay testing, regression checks, quest verification, and interactive software operation. The most transferable lesson is methodological: build around the real interface, train across relevant interaction distributions, preserve task state explicitly, and evaluate under live timing rather than only paused environments. The demonstrated results are promising for controlled digital worlds, while broader physical-world generalization remains a direction for future work rather than an established capability.
Sources
AI-assisted research summary. Check the original paper for complete methods, results, and limitations.