# Question Answering on HotPotQA (Success Rate and Best Delta) Source: [Question Answering on HotPotQA (Success Rate and Best Delta)]() Collection index: [llms.txt]() Data updated: 2026-03-13T06:58:37.268Z Evaluation of model performance on the HotpotQA dataset for Question Answering, measured using Success Rate (%) and Best Delta (pp). A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. | Scope | Recorded value | | --- | --- | | Task | Question Answering | | Dataset (including any stated split) | HotPotQA | | Metric | Success Rate | | Unit | PERCENT | | Direction | Higher is better | | Results loaded | 33 of 33 | | Coverage | All results currently returned for this benchmark | Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol. ## Metrics - [Success Rate](): Higher is better; unit: PERCENT. - [Best Delta (pp)](): Lower is better; unit: SCALAR. ## Recorded results | Method | Success Rate | Result date | Report paper | | --- | --- | --- | --- | | OPT | 82.7 | Not recorded | 68fa7bc61e209131cea8a5d4 | | LLaMA | 81 | Not recorded | 68fa7bc61e209131cea8a5d4 | | Structured Agent Distillation | 75.2 | Not recorded | 68fa7bc61e209131cea8a5d4 | | Structured Agent Distillation | 73.9 | Not recorded | 68fa7bc61e209131cea8a5d4 | | Token-level baseline | 71.5 | Not recorded | 68fa7bc61e209131cea8a5d4 | | GLM-4.7 | 70.3 | Not recorded | 6982d2a6a8b01e76cc8e9500 | | GLM-4.7 | 70.3 | Not recorded | 6982d2a6a8b01e76cc8e9500 | | Token-level baseline | 69.7 | Not recorded | 68fa7bc61e209131cea8a5d4 | | SeqKD | 69.2 | Not recorded | 68fa7bc61e209131cea8a5d4 | | GLM-4.7 | 68.7 | Not recorded | 6982d2a6a8b01e76cc8e9500 | | KD | 67.5 | Not recorded | 68fa7bc61e209131cea8a5d4 | | GLM-4.7 | 67.3 | Not recorded | 6982d2a6a8b01e76cc8e9500 | | Structured Agent Distillation | 67 | Not recorded | 68fa7bc61e209131cea8a5d4 | | GLM-4.7 | 66.7 | Not recorded | 6982d2a6a8b01e76cc8e9500 | | SeqKD | 66 | Not recorded | 68fa7bc61e209131cea8a5d4 | | KD | 64.3 | Not recorded | 68fa7bc61e209131cea8a5d4 | | MiniMax-M2.1 | 64 | Not recorded | 6982d2a6a8b01e76cc8e9500 | | Token-level baseline | 62.9 | Not recorded | 68fa7bc61e209131cea8a5d4 | | Structured Agent Distillation | 58.5 | Not recorded | 68fa7bc61e209131cea8a5d4 | | SeqKD | 58 | Not recorded | 68fa7bc61e209131cea8a5d4 | | Qwen-3-8B | 57 | Not recorded | 6982d2a6a8b01e76cc8e9500 | | KD | 56.2 | Not recorded | 68fa7bc61e209131cea8a5d4 | | Qwen-3-8B | 54.7 | Not recorded | 6982d2a6a8b01e76cc8e9500 | | Token-level baseline | 54.1 | Not recorded | 68fa7bc61e209131cea8a5d4 | | Qwen-3-8B | 53.3 | Not recorded | 6982d2a6a8b01e76cc8e9500 | | Qwen-3-8B | 51.3 | Not recorded | 6982d2a6a8b01e76cc8e9500 | | SeqKD | 50.3 | Not recorded | 68fa7bc61e209131cea8a5d4 | | Qwen-3-8B | 49.7 | Not recorded | 6982d2a6a8b01e76cc8e9500 | | KD | 49 | Not recorded | 68fa7bc61e209131cea8a5d4 | | MiniMax-M2.1 | 38.5 | Not recorded | 6982d2a6a8b01e76cc8e9500 | | MiniMax-M2.1 | 38 | Not recorded | 6982d2a6a8b01e76cc8e9500 | | MiniMax-M2.1 | 36.7 | Not recorded | 6982d2a6a8b01e76cc8e9500 | | MiniMax-M2.1 | 34.7 | Not recorded | 6982d2a6a8b01e76cc8e9500 | ### OPT Success Rate: 82.7. Result ID: 69b3b59c47f34ce9249d2aab. [Reporting paper]() Recorded evaluation configuration: {"Role":"Teacher","Size":"13B"}. This record does not assert independent reproduction. ### LLaMA Success Rate: 81. Result ID: 69b3b59c47f34ce9249d2ab8. [Reporting paper]() Recorded evaluation configuration: {"Role":"Teacher","Size":"13B"}. This record does not assert independent reproduction. ### Structured Agent Distillation Success Rate: 75.2. Result ID: 69b3b59c47f34ce9249d2abc. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Teacher":"LLaMA-13B","Student Size":"7B"}. This record does not assert independent reproduction. ### Structured Agent Distillation Success Rate: 73.9. Result ID: 69b3b59c47f34ce9249d2ab7. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Teacher":"OPT-13B","Student Size":"6.7B"}. This record does not assert independent reproduction. ### Token-level baseline Success Rate: 71.5. Result ID: 69b3b59c47f34ce9249d2abb. [Reporting paper]() Recorded evaluation configuration: {"Teacher":"LLaMA-13B","Student Size":"7B"}. This record does not assert independent reproduction. ### GLM-4.7 Success Rate: 70.3. Result ID: 699650455698665ec4950d91. [Reporting paper]() Recorded evaluation configuration: {"Intervention Mechanism":"Baseline","Calibration":"N/A"}. This record does not assert independent reproduction. ### GLM-4.7 Success Rate: 70.3. Result ID: 699650455698665ec4950d92. [Reporting paper]() Recorded evaluation configuration: {"Intervention Mechanism":"Rollback","Calibration":"Uncalibrated"}. This record does not assert independent reproduction. ### Token-level baseline Success Rate: 69.7. Result ID: 69b3b59c47f34ce9249d2ab6. [Reporting paper]() Recorded evaluation configuration: {"Teacher":"OPT-13B","Student Size":"6.7B"}. This record does not assert independent reproduction. ### SeqKD Success Rate: 69.2. Result ID: 69b3b59c47f34ce9249d2aba. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Teacher":"LLaMA-13B","Student Size":"7B"}. This record does not assert independent reproduction. ### GLM-4.7 Success Rate: 68.7. Result ID: 699650455698665ec4950d93. [Reporting paper]() Recorded evaluation configuration: {"Intervention Mechanism":"Rollback","Calibration":"Calibrated"}. This record does not assert independent reproduction. ### KD Success Rate: 67.5. Result ID: 69b3b59c47f34ce9249d2ab9. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Teacher":"LLaMA-13B","Student Size":"7B"}. This record does not assert independent reproduction. ### GLM-4.7 Success Rate: 67.3. Result ID: 699650455698665ec4950d94. [Reporting paper]() Recorded evaluation configuration: {"Intervention Mechanism":"Append","Calibration":"Uncalibrated"}. This record does not assert independent reproduction. ### Structured Agent Distillation Success Rate: 67. Result ID: 69b3b59c47f34ce9249d2ab3. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Teacher":"OPT-13B","Student Size":"2.7B"}. This record does not assert independent reproduction. ### GLM-4.7 Success Rate: 66.7. Result ID: 699650455698665ec4950d95. [Reporting paper]() Recorded evaluation configuration: {"Intervention Mechanism":"Append","Calibration":"Calibrated"}. This record does not assert independent reproduction. ### SeqKD Success Rate: 66. Result ID: 69b3b59c47f34ce9249d2ab5. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Teacher":"OPT-13B","Student Size":"6.7B"}. This record does not assert independent reproduction. ### KD Success Rate: 64.3. Result ID: 69b3b59c47f34ce9249d2ab4. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Teacher":"OPT-13B","Student Size":"6.7B"}. This record does not assert independent reproduction. ### MiniMax-M2.1 Success Rate: 64. Result ID: 699650455698665ec4950d96. [Reporting paper]() Recorded evaluation configuration: {"Intervention Mechanism":"Baseline","Calibration":"N/A"}. This record does not assert independent reproduction. ### Token-level baseline Success Rate: 62.9. Result ID: 69b3b59c47f34ce9249d2ab2. [Reporting paper]() Recorded evaluation configuration: {"Teacher":"OPT-13B","Student Size":"2.7B"}. This record does not assert independent reproduction. ### Structured Agent Distillation Success Rate: 58.5. Result ID: 69b3b59c47f34ce9249d2aaf. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Teacher":"OPT-13B","Student Size":"1.3B"}. This record does not assert independent reproduction. ### SeqKD Success Rate: 58. Result ID: 69b3b59c47f34ce9249d2ab1. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Teacher":"OPT-13B","Student Size":"2.7B"}. This record does not assert independent reproduction. ### Qwen-3-8B Success Rate: 57. Result ID: 699650455698665ec4950d8c. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Intervention Mechanism":"Baseline","Calibration":"N/A"}. This record does not assert independent reproduction. ### KD Success Rate: 56.2. Result ID: 69b3b59c47f34ce9249d2ab0. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Teacher":"OPT-13B","Student Size":"2.7B"}. This record does not assert independent reproduction. ### Qwen-3-8B Success Rate: 54.7. Result ID: 699650455698665ec4950d8e. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Intervention Mechanism":"Rollback","Calibration":"Calibrated"}. This record does not assert independent reproduction. ### Token-level baseline Success Rate: 54.1. Result ID: 69b3b59c47f34ce9249d2aae. [Reporting paper]() Recorded evaluation configuration: {"Teacher":"OPT-13B","Student Size":"1.3B"}. This record does not assert independent reproduction. ### Qwen-3-8B Success Rate: 53.3. Result ID: 699650455698665ec4950d8d. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Intervention Mechanism":"Rollback","Calibration":"Uncalibrated"}. This record does not assert independent reproduction. ### Qwen-3-8B Success Rate: 51.3. Result ID: 699650455698665ec4950d8f. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Intervention Mechanism":"Append","Calibration":"Uncalibrated"}. This record does not assert independent reproduction. ### SeqKD Success Rate: 50.3. Result ID: 69b3b59c47f34ce9249d2aad. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Teacher":"OPT-13B","Student Size":"1.3B"}. This record does not assert independent reproduction. ### Qwen-3-8B Success Rate: 49.7. Result ID: 699650455698665ec4950d90. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Intervention Mechanism":"Append","Calibration":"Calibrated"}. This record does not assert independent reproduction. ### KD Success Rate: 49. Result ID: 69b3b59c47f34ce9249d2aac. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Teacher":"OPT-13B","Student Size":"1.3B"}. This record does not assert independent reproduction. ### MiniMax-M2.1 Success Rate: 38.5. Result ID: 699650455698665ec4950d98. [Reporting paper]() Recorded evaluation configuration: {"Intervention Mechanism":"Rollback","Calibration":"Calibrated"}. This record does not assert independent reproduction. ### MiniMax-M2.1 Success Rate: 38. Result ID: 699650455698665ec4950d97. [Reporting paper]() Recorded evaluation configuration: {"Intervention Mechanism":"Rollback","Calibration":"Uncalibrated"}. This record does not assert independent reproduction. ### MiniMax-M2.1 Success Rate: 36.7. Result ID: 699650455698665ec4950d9a. [Reporting paper]() Recorded evaluation configuration: {"Intervention Mechanism":"Append","Calibration":"Calibrated"}. This record does not assert independent reproduction. ### MiniMax-M2.1 Success Rate: 34.7. Result ID: 699650455698665ec4950d99. [Reporting paper]() Recorded evaluation configuration: {"Intervention Mechanism":"Append","Calibration":"Uncalibrated"}. This record does not assert independent reproduction. Page 1 of 1. This is an index page, not the full collection.