# Aligning Flow Map Policies with Optimal Q-Guidance

Source: [Aligning Flow Map Policies with Optimal Q-Guidance](<https://www.sota2.com/research/paper/6a0405e6dbc836b87d827970>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2026-05-14T05:00:13.174Z

Authors: Christos Ziakas, Alessandra Russo, Avishek Joey Bose

Published: 2026-05-12T17:12:29.000Z

## Abstract

Generative policies based on expressive model classes, such as diffusion and flow matching, are well-suited to complex control problems with highly multimodal action distributions. Their expressivity, however, comes at a significant inference cost: generating each action typically requires simulating many steps of the generative process, compounding latency across sequential decision-making rollouts. We introduce flow map policies, a novel class of generative policies designed for fast action generation by learning to take arbitrary-size jumps including one-step jumps-across the generative dynamics of existing flow-based policies. We instantiate flow map policies for offline-to-online reinforcement learning (RL) and formulate online adaptation as a trust-region optimization problem that improves the critic's Q-value while remaining close to the offline policy. We theoretically derive FLOW MAP Q-GUIDANCE (FMQ), a principled closed-form learning target that is optimal for adapting offline flow map policies under a critic-guided trust-region constraint. We further introduce Q-GUIDED BEAM SEARCH (QGBS), a stochastic flow-map sampler that combines renoising with beam search to enable iterative inference-time refinement. Across 12 challenging robotic manipulation and locomotion tasks from OGBench and RoboMimic, FMQ achieves state-of-the-art performance in offline-to-online RL, outperforming the previous one-step policy MVP by a relative improvement of 21.3% on the average success rate.

## Recorded benchmark results

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

- [Robotic Control on RoboMimic & OGBench (Online Evaluation)](<https://www.sota2.com/research/sota/robotic-control-on-robomimic-and-ogbench-online-evaluation.md>): Robotic Control; RoboMimic & OGBench online evaluation; Can Success Rate: 97.

## Original sources and implementation

[Original paper](<https://arxiv.org/abs/2605.12416>)

[PDF](<https://arxiv.org/pdf/2605.12416.pdf>)