# RAG-RL: Advancing Retrieval-Augmented Generation via RL and Curriculum Learning Source: [RAG-RL: Advancing Retrieval-Augmented Generation via RL and Curriculum Learning]() Collection index: [llms.txt]() Data updated: 2026-04-30T05:00:04.703Z Authors: Jerry Huang, Siddarth Madala, Risham Sidhu, Cheng Niu, Hao Peng, Julia Hockenmaier, Tong Zhang Published: 2025-03-17T02:53:42.000Z ## Abstract Retrieval-augmented generation (RAG) systems rely on retrieval models for identifying relevant contexts and answer generation models for utilizing those contexts. However, retrievers exhibit imperfect recall and precision, limiting downstream performance. We introduce RAG-RL, an answer generation model trained not only to produce answers but also to identify and cite relevant information from larger sets of retrieved contexts, shifting some of the burden of identifying relevant documents from the retriever to the answer generator. Our approach uses curriculum learning, where the model is first trained on easier examples that include only relevant contexts. Our experiments show that these training samples enable models to acquire citation and reasoning skills with greater sample efficiency and generalizability, demonstrating strong model performance even as the number of irrelevant passages increases. We benchmark our methods on three open-domain multi-hop question answering datasets and report significant gains in answer and citation accuracy. Our experiments provide empirical insights into how easier training samples can give models stronger signals for learning specific skills (e.g., citation generation) and how different components of post-training (e.g., training set construction, rule-based rewards, training sample ordering, etc.) impact final model performance. ## Recorded benchmark results A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. - [Multi-hop Question Answering on HotpotQA (Answer F1, Citation F1, Joint F1)](): Open-domain multi-hop question answering; HotpotQA; Answer F1: 76.18. - [Open-domain multi-hop question answering on MuSiQue answerable](): Open-domain multi-hop question answering; MuSiQue answerable; Answer F1: 65.06. - [Open-domain multi-hop question answering on 2Wiki](): Open-domain multi-hop question answering; 2Wiki; Answer F1: 75.06. ## Original sources and implementation [Original paper]() [PDF]()