# Question Answering on HotpotQA (test) (Unreliability Rate Metric) Source: [Question Answering on HotpotQA (test) (Unreliability Rate Metric)]() Collection index: [llms.txt]() Data updated: 2026-06-30T09:21:57.137Z Evaluation of model performance on the test split of the HotpotQA dataset for Question Answering, specifically reporting the Unreliability Rate. A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. | Scope | Recorded value | | --- | --- | | Task | Question Answering | | Dataset (including any stated split) | HotpotQA (test) | | Metric | Unreliability Rate | | Unit | PERCENT | | Direction | Lower is better | | Results loaded | 28 of 28 | | Coverage | All results currently returned for this benchmark | Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol. ## Metrics - [Unreliability Rate](): Lower is better; unit: PERCENT. ## Recorded results | Method | Unreliability Rate | Result date | Report paper | | --- | --- | --- | --- | | KbSD | 38.8 | Not recorded | 6a434da0ba73165dbb743dc3 | | KbSD | 41.6 | Not recorded | 6a434da0ba73165dbb743dc3 | | KbPO | 51.8 | Not recorded | 6a434da0ba73165dbb743dc3 | | ReACT | 57.19 | Not recorded | 6a434da0ba73165dbb743dc3 | | KbPO | 57.6 | Not recorded | 6a434da0ba73165dbb743dc3 | | TCRAG | 59.17 | Not recorded | 6a434da0ba73165dbb743dc3 | | Search-R1 | 61.11 | Not recorded | 6a434da0ba73165dbb743dc3 | | Search-R1 | 62.76 | Not recorded | 6a434da0ba73165dbb743dc3 | | ARPO | 63.52 | Not recorded | 6a434da0ba73165dbb743dc3 | | FL-RAG | 65.58 | Not recorded | 6a434da0ba73165dbb743dc3 | | ReACT | 65.63 | Not recorded | 6a434da0ba73165dbb743dc3 | | ReSearch | 66.04 | Not recorded | 6a434da0ba73165dbb743dc3 | | ReSearch | 69.61 | Not recorded | 6a434da0ba73165dbb743dc3 | | Mem1 | 70.02 | Not recorded | 6a434da0ba73165dbb743dc3 | | FS-RAG | 70.79 | Not recorded | 6a434da0ba73165dbb743dc3 | | COT | 70.9 | Not recorded | 6a434da0ba73165dbb743dc3 | | AEPO | 71.29 | Not recorded | 6a434da0ba73165dbb743dc3 | | FL-RAG | 73.22 | Not recorded | 6a434da0ba73165dbb743dc3 | | Base | 73.37 | Not recorded | 6a434da0ba73165dbb743dc3 | | IRCOT | 73.71 | Not recorded | 6a434da0ba73165dbb743dc3 | | FS-RAG | 74.15 | Not recorded | 6a434da0ba73165dbb743dc3 | | ARPO | 74.8 | Not recorded | 6a434da0ba73165dbb743dc3 | | IRCOT | 75.5 | Not recorded | 6a434da0ba73165dbb743dc3 | | Base | 75.92 | Not recorded | 6a434da0ba73165dbb743dc3 | | COT | 76.18 | Not recorded | 6a434da0ba73165dbb743dc3 | | TCRAG | 78.06 | Not recorded | 6a434da0ba73165dbb743dc3 | | Mem1 | 79.85 | Not recorded | 6a434da0ba73165dbb743dc3 | | AEPO | 86.15 | Not recorded | 6a434da0ba73165dbb743dc3 | ### KbSD Unreliability Rate: 38.8. Result ID: 6a438a9afbc373592c9578a5. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"Qwen2.5-7B","Paradigm":"RL-based Agentic RAG"}. This record does not assert independent reproduction. ### KbSD Unreliability Rate: 41.6. Result ID: 6a438a9afbc373592c957897. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"Qwen2.5-3B","Paradigm":"RL-based Agentic RAG"}. This record does not assert independent reproduction. ### KbPO Unreliability Rate: 51.8. Result ID: 6a438a9afbc373592c9578a4. [Reporting paper]() Recorded evaluation configuration: {"Backbone":"Qwen2.5-7B","Paradigm":"RL-based Agentic RAG"}. This record does not assert independent reproduction. ### ReACT Unreliability Rate: 57.19. Result ID: 6a438a9afbc373592c95789c. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Backbone":"Qwen2.5-7B","Paradigm":"Agentic RAG"}. This record does not assert independent reproduction. ### KbPO Unreliability Rate: 57.6. Result ID: 6a438a9afbc373592c957896. [Reporting paper]() Recorded evaluation configuration: {"Backbone":"Qwen2.5-3B","Paradigm":"RL-based Agentic RAG"}. This record does not assert independent reproduction. ### TCRAG Unreliability Rate: 59.17. Result ID: 6a438a9afbc373592c95789e. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"Qwen2.5-7B","Paradigm":"Agentic RAG"}. This record does not assert independent reproduction. ### Search-R1 Unreliability Rate: 61.11. Result ID: 6a438a9afbc373592c9578a0. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"Qwen2.5-7B","Paradigm":"RL-based Agentic RAG"}. This record does not assert independent reproduction. ### Search-R1 Unreliability Rate: 62.76. Result ID: 6a438a9afbc373592c957892. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"Qwen2.5-3B","Paradigm":"RL-based Agentic RAG"}. This record does not assert independent reproduction. ### ARPO Unreliability Rate: 63.52. Result ID: 6a438a9afbc373592c957894. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"Qwen2.5-3B","Paradigm":"RL-based Agentic RAG"}. This record does not assert independent reproduction. ### FL-RAG Unreliability Rate: 65.58. Result ID: 6a438a9afbc373592c95789b. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"Qwen2.5-7B","Paradigm":"Naive RAG"}. This record does not assert independent reproduction. ### ReACT Unreliability Rate: 65.63. Result ID: 6a438a9afbc373592c95788e. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Backbone":"Qwen2.5-3B","Paradigm":"Agentic RAG"}. This record does not assert independent reproduction. ### ReSearch Unreliability Rate: 66.04. Result ID: 6a438a9afbc373592c957891. [Reporting paper]() Recorded evaluation configuration: {"Backbone":"Qwen2.5-3B","Paradigm":"RL-based Agentic RAG"}. This record does not assert independent reproduction. ### ReSearch Unreliability Rate: 69.61. Result ID: 6a438a9afbc373592c95789f. [Reporting paper]() Recorded evaluation configuration: {"Backbone":"Qwen2.5-7B","Paradigm":"RL-based Agentic RAG"}. This record does not assert independent reproduction. ### Mem1 Unreliability Rate: 70.02. Result ID: 6a438a9afbc373592c9578a3. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"Qwen2.5-7B","Paradigm":"RL-based Agentic RAG"}. This record does not assert independent reproduction. ### FS-RAG Unreliability Rate: 70.79. Result ID: 6a438a9afbc373592c95789a. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Backbone":"Qwen2.5-7B","Paradigm":"Naive RAG"}. This record does not assert independent reproduction. ### COT Unreliability Rate: 70.9. Result ID: 6a438a9afbc373592c957899. [Reporting paper]() Recorded evaluation configuration: {"Backbone":"Qwen2.5-7B","Paradigm":"No RAG"}. This record does not assert independent reproduction. ### AEPO Unreliability Rate: 71.29. Result ID: 6a438a9afbc373592c957893. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"Qwen2.5-3B","Paradigm":"RL-based Agentic RAG"}. This record does not assert independent reproduction. ### FL-RAG Unreliability Rate: 73.22. Result ID: 6a438a9afbc373592c95788d. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"Qwen2.5-3B","Paradigm":"Naive RAG"}. This record does not assert independent reproduction. ### Base Unreliability Rate: 73.37. Result ID: 6a438a9afbc373592c957898. [Reporting paper]() Recorded evaluation configuration: {"Backbone":"Qwen2.5-7B","Paradigm":"No RAG"}. This record does not assert independent reproduction. ### IRCOT Unreliability Rate: 73.71. Result ID: 6a438a9afbc373592c95789d. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Backbone":"Qwen2.5-7B","Paradigm":"Agentic RAG"}. This record does not assert independent reproduction. ### FS-RAG Unreliability Rate: 74.15. Result ID: 6a438a9afbc373592c95788c. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Backbone":"Qwen2.5-3B","Paradigm":"Naive RAG"}. This record does not assert independent reproduction. ### ARPO Unreliability Rate: 74.8. Result ID: 6a438a9afbc373592c9578a2. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"Qwen2.5-7B","Paradigm":"RL-based Agentic RAG"}. This record does not assert independent reproduction. ### IRCOT Unreliability Rate: 75.5. Result ID: 6a438a9afbc373592c95788f. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Backbone":"Qwen2.5-3B","Paradigm":"Agentic RAG"}. This record does not assert independent reproduction. ### Base Unreliability Rate: 75.92. Result ID: 6a438a9afbc373592c95788a. [Reporting paper]() Recorded evaluation configuration: {"Backbone":"Qwen2.5-3B","Paradigm":"No RAG"}. This record does not assert independent reproduction. ### COT Unreliability Rate: 76.18. Result ID: 6a438a9afbc373592c95788b. [Reporting paper]() Recorded evaluation configuration: {"Backbone":"Qwen2.5-3B","Paradigm":"No RAG"}. This record does not assert independent reproduction. ### TCRAG Unreliability Rate: 78.06. Result ID: 6a438a9afbc373592c957890. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"Qwen2.5-3B","Paradigm":"Agentic RAG"}. This record does not assert independent reproduction. ### Mem1 Unreliability Rate: 79.85. Result ID: 6a438a9afbc373592c957895. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"Qwen2.5-3B","Paradigm":"RL-based Agentic RAG"}. This record does not assert independent reproduction. ### AEPO Unreliability Rate: 86.15. Result ID: 6a438a9afbc373592c9578a1. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"Qwen2.5-7B","Paradigm":"RL-based Agentic RAG"}. This record does not assert independent reproduction. Page 1 of 1. This is an index page, not the full collection.