Papers
arxiv:2611.18397

Gamer: A Generalist Vision-Language-Action Foundation Model for Long-Horizon Real-Time Game Agent

Published on Nov 13
ยท Submitted by
Kuan Zhang
on Nov 16
#2 Paper of the day
Authors:
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,

Abstract

Gamer is a generalist agent framework that integrates perception, memory, and multi-step reasoning, trained through continued pre-training, sparse-thinking instruction tuning, and multi-turn reinforcement learning to interact efficiently in 2D and 3D environments.

We introduce Gamer, a generalist agent framework built for long-horizon tasks in complex interactive environments. Its end-to-end architecture combines perception, memory management, and multi-step reasoning. A shared training and deployment pipeline supports interaction with both conventional 2D graphical interfaces and complex 3D simulated worlds. We first continue pre-training on a dataset constructed from tens of thousands of hours of device-level keyboard and mouse interaction. These trajectories span simulated games and embodied environments, allowing the model to learn environmental dynamics across both physical and virtual domains. Sparse-thinking instruction tuning then substantially reduces redundant computation while preserving the reasoning needed for complex decisions. We train the policy for long-horizon interaction with a stable multi-turn reinforcement learning method that addresses finite context windows, accumulated rollout errors, and sparse rewards. For real-time interaction, a cloud-edge inference engine serves numerous simulated game environments concurrently, supports unbounded-context execution, and keeps online inference latency low. Across multiple challenging 3D open-world games, Gamer demonstrates solid generalization and operational efficiency comparable to skilled players. It also achieves competitive results on a range of conventional 2D game benchmarks. These results suggest that even repetitive keyboard and mouse trajectories can support substantial gains from scaling data and learning algorithms. Training on these trajectories helps Gamer learn complex skills more steadily in environments with both physical and digital properties. This approach offers a path toward generalist agents that can interact across digital and physical worlds.

Community

Paper author Paper submitter

๐ŸŽฎ Can AI truly learn to play complex 3D games like humansโ€”just by watching the screen and using a keyboard and mouse?

Weโ€™re excited to introduce Gamer, a generalist Vision-Language-Action foundation model for long-horizon, real-time game interaction, developed by the miHoYo Honkai AI R&D team!

Unlike game-specific agents, Gamer interacts with the world through the same interface humans use: screenshots, keyboard, and mouse. No privileged game states or game-specific APIs!

To make this possible, we built a complete training and inference pipeline, from large-scale pre-training to real-time reinforcement learning.

๐Ÿ”ฅ 1. Scaling Game Intelligence. We leverage 10K+ hours of gameplay trajectories for continual pre-training (CPT) and systematically study scaling laws across model sizes and data mixtures, exploring how generalizable game-playing abilities emerge through scaling.

๐Ÿง  2. Teaching Agents to Think and Remember. We build a multi-level, high-quality automated annotation pipeline for gameplay data, producing ~1.2M reasoning annotations and ~223K memory annotations. Our SFT framework teaches agents when to reason, what to remember, and how to execute long-horizon tasks.

โšก 3. Making Real-Time Interaction Truly Real-Time. We develop Streaming-vLLM, a high-concurrency inference engine optimized for continuous game interaction. Through streaming KV-cache reuse, speculative decoding, and system-level optimizations, we achieve over 20ร— lower inference latency in our benchmark setup, enabling real-time keyboard and mouse control.

๐Ÿš€ 4. Scaling RL to Hour-Long Interactions. We build large-scale RL infrastructure supporting hour-long, real-time interaction trajectories in complex game environments. Our Temporal-Aware GRPO (TA-GRPO) enables learning from long-horizon environment feedback, helping agents continuously improve their planning and execution abilities.

๐ŸŽฏ 5. Beyond Benchmarks: Can AI Actually Play Through Games? We conduct extensive automated testing in large-scale 3D games. Across multiple main storylines in Honkai: Star Rail, Gamer successfully handles over 95% of the challenges! We also evaluate diverse side quests, navigation tasks, and challenging puzzles, demonstrating strong performance across different types of interactions.

๐Ÿ’ก One finding we find especially exciting: Faster inference doesn't just make agents respond faster. It also makes online RL training more effective! Real-time interaction plays a critical role in both learning and deployment.

๐ŸŒ Why does this matter?

We believe games are more than entertainment or static benchmarks. They are rich, dynamic environments where agents must continuously perceive, reason, remember, plan, and act.

Gamer represents a step toward building general-purpose interactive agents, with exciting potential for automated game testing, embodied intelligence, and long-horizon autonomous systems.

Our goal isn't just to teach AI how to play games. It's to explore how AI can learn to understand, interact with, and eventually master complex virtual worlds.

๐ŸŽฌ Project Page & Gameplay Demos: https://mihoyo-gamer.github.io/

๐Ÿ“„ Paper: https://arxiv.org/abs/2611.18397

๐Ÿ’ฌ We'd love to hear your thoughts, questions, and feedback from the community!

If you find this direction interesting, we'd really appreciate your support on Hugging Face Daily Papers! ๐Ÿš€

Sign up or log in to comment

Get this paper in your agent:

hf papers read 2611.18397
Don't have the latest CLI?
curl -LsSf https://hf.co/cli/install.sh | bash

Models citing this paper 0

No model linking this paper

Cite arxiv.org/abs/2611.18397 in a model README.md to link it from this page.

Datasets citing this paper 0

No dataset linking this paper

Cite arxiv.org/abs/2611.18397 in a dataset README.md to link it from this page.

Spaces citing this paper 0

No Space linking this paper

Cite arxiv.org/abs/2611.18397 in a Space README.md to link it from this page.

Collections including this paper 0

No Collection including this paper

Add this paper to a collection to link it from this page.