# Document Optimization for Black-Box Retrieval via Reinforcement Learning Source: [Document Optimization for Black-Box Retrieval via Reinforcement Learning]() Collection index: [llms.txt]() Data updated: 2026-08-07T05:00:01.339Z Authors: Omri Uzan, Ron Polonsky, Douwe Kiela, Christopher Potts Published: 2026-04-06T18:41:40.000Z ## Abstract Document expansion is a classical technique for improving retrieval quality, and is attractive since it shifts computation offline, avoiding additional query-time processing. However, when applied to modern retrievers, it has been shown to degrade performance, often introducing noise that obfuscates the discriminative signal. We recast document expansion as a document optimization problem: a language model or a vision language model is fine-tuned to transform documents into representations that better align with the expected query distribution under a target retriever, using GRPO with the retriever's ranking improvements as rewards. This approach requires only black-box access to retrieval ranks, and is applicable across single-vector, multi-vector and lexical retrievers. We evaluate our approach on code retrieval and visual document retrieval (VDR) tasks. We find that learned document transformations yield retrieval gains and in many settings enable smaller, more efficient retrievers to outperform larger ones. For example, applying document optimization to OpenAI text-embedding-3-small model improves nDCG5 on code (58.7 to 66.8) and VDR (53.3 to 57.6), even slightly surpassing the 6.5X more expensive OpenAI text-embedding-3-large model (66.3 on code; 57.0 on VDR). When retriever weights are accessible, document optimization is often competitive with fine-tuning, and in some settings their combination performs best, improving Jina-ColBERT-V2 from 55.8 to 63.3 on VDR and from 48.6 to 61.8 on code retrieval. ## Recorded benchmark results A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. - [Code Retrieval on HumanEval (val)](): Code Retrieval; HumanEval (val); nDCG@5: 98.5. - [Code Retrieval on DS10K (val)](): Code Retrieval; DS10K (val); nDCG@5: 63.8. - [Visual Document Retrieval on ViDoRe Biomed 2 (val)](): Visual document retrieval; ViDoRe Biomed 2 (val); nDCG@5: 61.2. - [Visual Document Retrieval on ViDoRe2 Econ (val)](): Visual document retrieval; ViDoRe2 Econ (val); nDCG@5: 61. - [Visual Document Retrieval on ViDoRe2 ESG Human (val)](): Visual document retrieval; ViDoRe2 ESG Human (val); nDCG@5: 72.6. - [Visual Document Retrieval on ViDoRe ESG Full 2 (val)](): Visual document retrieval; ViDoRe ESG Full 2 (val); nDCG@5: 63.1. - [Visual Document Retrieval on ViDoRe2 Average (val)](): Visual document retrieval; ViDoRe2 Average (val); nDCG@5: 63.3. - [Code Retrieval on MBPP (val)](): Code Retrieval; MBPP (val); nDCG@5: 91.6. - [Code Retrieval on MTEB Code (val)](): Code Retrieval; MTEB Code Retrieval Average (val); nDCG@5: 73.5. - [Code Retrieval on FreshStack (val)](): Code Retrieval; FreshStack (val); nDCG@5: 40.3. ## Original sources and implementation [Original paper]() [PDF]()