Do LLM Agents Still Need to Be Told How to Self-Improve?
Abstract
Capable frozen LLM agents can autonomously construct context-based self-learning algorithms from basic resources and a general goal, yielding text artifacts that transfer to new game and 3D navigation tasks.
LLM agents increasingly drive their own learning. Self-improvement methods delegate various components of model optimization to the models themselves. As models become more capable, this motivates a basic question: do these increasingly capable models still need us to specify how they should self-improve? In this paper we study an autonomous paradigm in which, given basic primitives, resources, and a general goal for self-improvement, agents construct their own self-learning algorithm for context-based optimization. Using the setting of gameplay as a testbed, a frozen LLM agent is given a compute budget and tools for collecting experience and updating text documents, and allowed to freely experiment without an externally imposed loop or access to real-world data. We find that sufficiently capable models do self-improve under these conditions, demonstrating strategies that align with classic cognitive frameworks for self-regulated learning and constructs from machine learning literature on self-generated curricula. Furthermore, we study the extent to which the learned text artifacts transfer to external benchmarks, observing improvement across different game horizons, mechanics, and 3D navigation.
Community
TL;DR: We give an LLM agent a general self-improvement goal, a budget, and basic tools for collecting experience and editing text. We do not give it real tasks or an explicit recipe. Sufficiently capable models successfully compose their own self-improvement loops using human-recognizable learning strategies.
The result that a capable frozen agent builds its own self-learning loop from just tools and a budget is intriguing. I wonder how much of the gain comes from the text documents it writes versus the experiments it chooses to run. Did you compare the transferred documents against ones written by a fixed, hand-designed loop?
This is an automated message from the Librarian Bot. I found the following papers similar to this paper.
The following papers were recommended by the Semantic Scholar API
- Flex-RSI: Benchmarking Recursive Self-Improvement of Frontier Agents (2026)
- Recursive self-improvement of AI research agents (2026)
- RRSI: Regularized Recursive Self-Improvement of Agent Harnesses (2026)
- Toward Recursive Self-Improvement in Physical AI (2026)
- RSI-Env: An Open-Ended Environment for Recursive Self-Improvement of Agents (2026)
- Learn2Play Bench: How Well Do LLM Agents Learn from Experience in Unfamiliar Environments? (2026)
- Learning Meta-Skills for Agent Harness Design in Test-Time AI4AI (2026)
Please give a thumbs up to this comment if you found it helpful!
If you want recommendations for any Paper on Hugging Face checkout this Space
You can directly ask Librarian Bot for paper recommendations by tagging it in a comment: @librarian-bot recommend
Get this paper in your agent:
hf papers read 2611.23219 Don't have the latest CLI?
curl -LsSf https://hf.co/cli/install.sh | bash Models citing this paper 0
No model linking this paper
Datasets citing this paper 0
No dataset linking this paper
Spaces citing this paper 0
No Space linking this paper
Collections including this paper 0
No Collection including this paper