VGAS+: Value-Guided Action-Chunk Selection and Refinement

Abstract

Vision–Language–Action (VLA) models bridge multimodal reasoning with physical control, but adapting them to new tasks with scarce demonstrations remains unreliable. While few-shot SFT policies often produce semantically plausible action chunks, small geometric errors can lead to divergent execution outcomes. We study few-shot VLA adaptation from a value-guided policy improvement perspective, learning a long-horizon critic over temporally extended action chunks. At the core of our approach is the Q-Chunk-Former, a geometrically grounded Transformer critic that maps vision–language observations and proprioceptive states to chunk-level values. To train this critic under scarce demonstrations, we introduce Explicit Geometric Regularization (EGR), which shapes a discriminative value landscape and preserves ranking resolution among near-miss action candidates. Building on this critic, VGAS performs inference-time Best-of-N selection by keeping the SFT policy fixed and executing the candidate chunk with the highest predicted return. We then introduce VGAS+, which extends VGAS with post-training policy extraction: it optimizes candidate chunks in action space and distills the critic-refined targets into the VLA through its native supervised generative objective, avoiding direct critic backpropagation through the flow-matching-based action head. Experiments on LIBERO and MetaWorld, together with theoretical analysis, show that VGAS improves fixed-policy execution, while VGAS+ further outperforms evaluated SFT and offline RL baselines under limited demonstrations.