# GAPD: Gold-Action Policy Distillation for Agentic Reinforcement Learning in Knowledge Base Question Answering Source: [GAPD: Gold-Action Policy Distillation for Agentic Reinforcement Learning in Knowledge Base Question Answering]() Collection index: [llms.txt]() Data updated: 2026-06-05T05:00:04.417Z Authors: Xin Sun, Jianan Xie, Zhongqi Chen, Qiang Liu, Shu Wu, Bowen Song, Weiqiang Wang, Zilei Wang, Liang Wang Published: 2026-05-28T08:28:10.000Z ## Abstract Reinforcement learning (RL) is a natural fit for agentic knowledge base question answering (KBQA), where a model must issue executable actions, observe knowledge-base feedback, and eventually return an answer. However, current RL-based KBQA systems mainly optimize sparse rewards from the final answer, leaving intermediate action errors weakly supervised. This is especially limiting for logical-form annotated KBQA benchmarks: gold logical forms can be converted into executable action sequences, but existing pipelines use them mainly for warm-start data construction rather than for on-policy RL updates. We propose GAPD, a training-time Gold-Action Policy Distillation framework that adds dense token-level guidance to outcome-based RL. To align gold actions with on-policy student rollouts, GAPD uses MID-ANCHOR MATCHING: it treats the intermediate entities reached during student exploration and gold execution as state anchors, and matches student states to gold states through these explored entity sets. The current policy conditioned on this aligned gold action serves as a stop-gradient teacher, whose token distribution is distilled back to the ordinary student policy over generated action-token spans. GAPD consistently surpasses the current state of the art on WebQSP, GrailQA, and GraphQ. ## Recorded benchmark results A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. - [Knowledge Base Question Answering on GrailQA (dev)](): Knowledge Base Question Answering; GrailQA (dev); Overall F1: 89.5. - [Knowledge Base Question Answering on GrailQA i.i.d. (dev)](): Knowledge Base Question Answering; GrailQA i.i.d. (dev); F1: 93.8. - [Knowledge Base Question Answering on GrailQA Compositional (dev)](): Knowledge Base Question Answering; GrailQA Compositional (dev); F1 Score: 87. - [Knowledge Base Question Answering on GraphQ (test)](): Knowledge Base Question Answering; GraphQ (test); F1: 55.1. - [Knowledge Base Question Answering on WebQSP (test)](): Knowledge Base Question Answering; WEBQSP (test); Hit@1: not recorded. ## Original sources and implementation [Original paper]() [PDF]()