Gamer: A Generalist Vision-Language-Action Foundation Model for Long-Horizon Real-Time Game Agent
Abstract
Gamer is a generalist agent framework that integrates perception, memory, and multi-step reasoning, trained through continued pre-training, sparse-thinking instruction tuning, and multi-turn reinforcement learning to interact efficiently in 2D and 3D environments.
We introduce Gamer, a generalist agent framework built for long-horizon tasks in complex interactive environments. Its end-to-end architecture combines perception, memory management, and multi-step reasoning. A shared training and deployment pipeline supports interaction with both conventional 2D graphical interfaces and complex 3D simulated worlds. We first continue pre-training on a dataset constructed from tens of thousands of hours of device-level keyboard and mouse interaction. These trajectories span simulated games and embodied environments, allowing the model to learn environmental dynamics across both physical and virtual domains. Sparse-thinking instruction tuning then substantially reduces redundant computation while preserving the reasoning needed for complex decisions. We train the policy for long-horizon interaction with a stable multi-turn reinforcement learning method that addresses finite context windows, accumulated rollout errors, and sparse rewards. For real-time interaction, a cloud-edge inference engine serves numerous simulated game environments concurrently, supports unbounded-context execution, and keeps online inference latency low. Across multiple challenging 3D open-world games, Gamer demonstrates solid generalization and operational efficiency comparable to skilled players. It also achieves competitive results on a range of conventional 2D game benchmarks. These results suggest that even repetitive keyboard and mouse trajectories can support substantial gains from scaling data and learning algorithms. Training on these trajectories helps Gamer learn complex skills more steadily in environments with both physical and digital properties. This approach offers a path toward generalist agents that can interact across digital and physical worlds.
Community
๐ฎ Can AI truly learn to play complex 3D games like humansโjust by watching the screen and using a keyboard and mouse?
Weโre excited to introduce Gamer, a generalist Vision-Language-Action foundation model for long-horizon, real-time game interaction, developed by the miHoYo Honkai AI R&D team!
Unlike game-specific agents, Gamer interacts with the world through the same interface humans use: screenshots, keyboard, and mouse. No privileged game states or game-specific APIs!
To make this possible, we built a complete training and inference pipeline, from large-scale pre-training to real-time reinforcement learning.
๐ฅ 1. Scaling Game Intelligence. We leverage 10K+ hours of gameplay trajectories for continual pre-training (CPT) and systematically study scaling laws across model sizes and data mixtures, exploring how generalizable game-playing abilities emerge through scaling.
๐ง 2. Teaching Agents to Think and Remember. We build a multi-level, high-quality automated annotation pipeline for gameplay data, producing ~1.2M reasoning annotations and ~223K memory annotations. Our SFT framework teaches agents when to reason, what to remember, and how to execute long-horizon tasks.
โก 3. Making Real-Time Interaction Truly Real-Time. We develop Streaming-vLLM, a high-concurrency inference engine optimized for continuous game interaction. Through streaming KV-cache reuse, speculative decoding, and system-level optimizations, we achieve over 20ร lower inference latency in our benchmark setup, enabling real-time keyboard and mouse control.
๐ 4. Scaling RL to Hour-Long Interactions. We build large-scale RL infrastructure supporting hour-long, real-time interaction trajectories in complex game environments. Our Temporal-Aware GRPO (TA-GRPO) enables learning from long-horizon environment feedback, helping agents continuously improve their planning and execution abilities.
๐ฏ 5. Beyond Benchmarks: Can AI Actually Play Through Games? We conduct extensive automated testing in large-scale 3D games. Across multiple main storylines in Honkai: Star Rail, Gamer successfully handles over 95% of the challenges! We also evaluate diverse side quests, navigation tasks, and challenging puzzles, demonstrating strong performance across different types of interactions.
๐ก One finding we find especially exciting: Faster inference doesn't just make agents respond faster. It also makes online RL training more effective! Real-time interaction plays a critical role in both learning and deployment.
๐ Why does this matter?
We believe games are more than entertainment or static benchmarks. They are rich, dynamic environments where agents must continuously perceive, reason, remember, plan, and act.
Gamer represents a step toward building general-purpose interactive agents, with exciting potential for automated game testing, embodied intelligence, and long-horizon autonomous systems.
Our goal isn't just to teach AI how to play games. It's to explore how AI can learn to understand, interact with, and eventually master complex virtual worlds.
๐ฌ Project Page & Gameplay Demos: https://mihoyo-gamer.github.io/
๐ Paper: https://arxiv.org/abs/2611.18397
๐ฌ We'd love to hear your thoughts, questions, and feedback from the community!
If you find this direction interesting, we'd really appreciate your support on Hugging Face Daily Papers! ๐
Get this paper in your agent:
hf papers read 2611.18397 Don't have the latest CLI?
curl -LsSf https://hf.co/cli/install.sh | bash Models citing this paper 0
No model linking this paper
Datasets citing this paper 0
No dataset linking this paper
Spaces citing this paper 0
No Space linking this paper
Collections including this paper 0
No Collection including this paper