# In-Context Compositional Q-Learning for Offline Reinforcement Learning Source: [In-Context Compositional Q-Learning for Offline Reinforcement Learning]() Collection index: [llms.txt]() Data updated: 2026-03-20T05:00:04.947Z Authors: Qiushui Xu, Yuhao Huang, Yushu Jiang, Lei Song, Jinyu Wang, Wenliang Zheng, Jiang Bian Published: 2025-09-28T20:55:21.000Z ## Abstract Accurate estimation of the Q-function is a central challenge in offline reinforcement learning. However, existing approaches often rely on a shared global Q-function, which is inadequate for capturing the compositional structure of tasks that consist of diverse subtasks. We propose In-context Compositional Q-Learning (ICQL), an offline RL framework that formulates Q-learning as a contextual inference problem and uses linear Transformers to adaptively infer local Q-functions from retrieved transitions without explicit subtask labels. Theoretically, we show that, under two assumptions -- linear approximability of the local Q-function and accurate inference of weights from retrieved context -- ICQL achieves a bounded approximation error for the Q-function and enables near-optimal policy extraction. Empirically, ICQL substantially improves performance in offline settings, achieving gains of up to 16.4% on kitchen tasks and up to 8.8% and 6.3% on MuJoCo and Adroit tasks, respectively. These results highlight the underexplored potential of in-context learning for robust and compositional value estimation and establish ICQL as a principled and effective framework for offline RL. ## Recorded benchmark results A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. - [Offline Reinforcement Learning on D4RL (MuJoCo M/MR/ME & Adroit/Kitchen Subset)](): Offline Reinforcement Learning; D4RL; Walker2d (Medium Expert) Score: 113.3. - [Goal-conditioned Manipulation on D4RL Kitchen Tasks](): Goal-conditioned manipulation; D4RL Kitchen Tasks; Kitchen Complete v0 Success: 79.3. - [Locomotion on D4RL MuJoCo Tasks (Average across v2 Environments)](): Locomotion; D4RL MuJoCo Tasks; Average D4RL Locomotion Score (v2): 80.6. - [Dexterous Manipulation on D4RL Adroit Tasks](): Dexterous Manipulation; D4RL Adroit Tasks; Performance Score: not recorded. ## Original sources and implementation [Original paper]() [PDF]()