Papers
arxiv:2611.08698

VICTR: Value-Informed In-Context Learning for Vision-Language-Action Models

Published on Nov 5
Authors:
,
,
,
,
,
,
,

Abstract

VICTR improves in-context learning in vision-language-action models by using a progress estimator to retrieve demonstration chunks aligned with the current execution stage, enabling better adaptation to unseen real-world manipulation tasks.

Recent advances in robot foundation models have highlighted the potential of in-context learning (ICL) for adapting robot behavior from demonstrations without further fine-tuning. Previous work has used retrieval-based post-training to equip pretrained vision-language-action (VLA) models with the capability for ICL by teaching them to condition action predictions on retrieved visually-relevant demonstration chunks. However, the effectiveness of retrieval hinges on selecting context relevant to the current execution stage, which visual similarity alone may not capture. We introduce VICTR, a framework enabling in-context learning in VLAs via progress-informed context retrieval built around an In-Context Value Function Estimator (IC-VFE). IC-VFE conditions on a reference demonstration video, a language instruction and the current observation to estimate task-relative progress. By conditioning progress estimation on a reference demonstration, IC-VFE can align the current state with a demonstrated execution even for tasks unseen during training. VICTR uses the estimated progress to select demonstration chunks at corresponding stages of execution and supplies them to a π_{0.5} policy post-trained for in-context action generation. On unseen out-of-domain real-world manipulation tasks, VICTR raises the success rate from 6.2% for a fine-tuned π_{0.5} baseline to 23.8% and more than doubles the task progress rate, from 28.9% to 55.5%. Project page with code, data and videos: https://ai4ce.github.io/VICTR/

Community

Thank you for sharing your work. VICTR learns demonstration-conditioned progress estimates and uses them to retrieve relevant demonstration chunks for a VLA. Its stage-aware selection provides a concrete mechanism for choosing an example because it is useful at the current execution phase, rather than only visually similar.

We discuss your work in Section 3.1 (“Context-conditioned policies”), p. 19 of the updated manuscript of our survey, In-Context Learning for Robots: Methods and Applications. We discuss learned progress estimation as the basis for selecting demonstration chunks that match the current execution phase. Read the updated discussion.

If you find our discussion of demonstration retrieval for contextual control useful for your own work, we would be grateful if you would consider citing the survey.

Sign up or log in to comment

Get this paper in your agent:

hf papers read 2611.08698
Don't have the latest CLI?
curl -LsSf https://hf.co/cli/install.sh | bash

Models citing this paper 4

Datasets citing this paper 1

Spaces citing this paper 0

No Space linking this paper

Cite arxiv.org/abs/2611.08698 in a Space README.md to link it from this page.

Collections including this paper 0

No Collection including this paper

Add this paper to a collection to link it from this page.