# Search-based Question Answering on SearchQA Source: [Search-based Question Answering on SearchQA]() Collection index: [llms.txt]() Data updated: 2026-06-02T12:07:20.886Z Evaluation of Search-based Question Answering performance on the SearchQA dataset, reporting component scores for Hotpot, 2Wiki, and Overall results. A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. | Scope | Recorded value | | --- | --- | | Task | Search-based Question Answering | | Dataset (including any stated split) | SearchQA | | Metric | Hotpot Score | | Unit | PERCENT | | Direction | Higher is better | | Results loaded | 18 of 18 | | Coverage | All results currently returned for this benchmark | Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol. ## Metrics - [Hotpot Score](): Higher is better; unit: PERCENT. - [2Wiki Score](): Higher is better; unit: PERCENT. - [Overall Score](): Higher is better; unit: PERCENT. ## Recorded results | Method | Hotpot Score | Result date | Report paper | | --- | --- | --- | --- | | HarnessForge | 42 | Not recorded | 6a1e6435dbd7150506f8583d | | GRPO | 41.33 | Not recorded | 6a1e6435dbd7150506f8583d | | HarnessForge | 41.33 | Not recorded | 6a1e6435dbd7150506f8583d | | SFT | 40.67 | Not recorded | 6a1e6435dbd7150506f8583d | | RLOO | 40 | Not recorded | 6a1e6435dbd7150506f8583d | | SFT | 36.67 | Not recorded | 6a1e6435dbd7150506f8583d | | GRPO | 36 | Not recorded | 6a1e6435dbd7150506f8583d | | RLOO | 35.33 | Not recorded | 6a1e6435dbd7150506f8583d | | MermaidFlow | 32.67 | Not recorded | 6a1e6435dbd7150506f8583d | | MaAS | 32 | Not recorded | 6a1e6435dbd7150506f8583d | | MermaidFlow | 31.33 | Not recorded | 6a1e6435dbd7150506f8583d | | ADAS | 31.33 | Not recorded | 6a1e6435dbd7150506f8583d | | AFlow | 31.33 | Not recorded | 6a1e6435dbd7150506f8583d | | AFlow | 30.67 | Not recorded | 6a1e6435dbd7150506f8583d | | MaAS | 30.67 | Not recorded | 6a1e6435dbd7150506f8583d | | AgentSquare | 29.33 | Not recorded | 6a1e6435dbd7150506f8583d | | AgentSquare | 29.33 | Not recorded | 6a1e6435dbd7150506f8583d | | ADAS | 28.67 | Not recorded | 6a1e6435dbd7150506f8583d | ### HarnessForge Hotpot Score: 42. Result ID: 6a1ec734fbc373592c1a4aa8. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Backbone":"Qwen3-4B"}. This record does not assert independent reproduction. ### GRPO Hotpot Score: 41.33. Result ID: 6a1ec734fbc373592c1a4ab0. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Backbone":"Qwen3-8B"}. This record does not assert independent reproduction. ### HarnessForge Hotpot Score: 41.33. Result ID: 6a1ec734fbc373592c1a4ab1. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Backbone":"Qwen3-8B"}. This record does not assert independent reproduction. ### SFT Hotpot Score: 40.67. Result ID: 6a1ec734fbc373592c1a4aae. [Reporting paper]() Recorded evaluation configuration: {"Backbone":"Qwen3-8B"}. This record does not assert independent reproduction. ### RLOO Hotpot Score: 40. Result ID: 6a1ec734fbc373592c1a4aaf. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"Qwen3-8B"}. This record does not assert independent reproduction. ### SFT Hotpot Score: 36.67. Result ID: 6a1ec734fbc373592c1a4aa5. [Reporting paper]() Recorded evaluation configuration: {"Backbone":"Qwen3-4B"}. This record does not assert independent reproduction. ### GRPO Hotpot Score: 36. Result ID: 6a1ec734fbc373592c1a4aa7. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Backbone":"Qwen3-4B"}. This record does not assert independent reproduction. ### RLOO Hotpot Score: 35.33. Result ID: 6a1ec734fbc373592c1a4aa6. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"Qwen3-4B"}. This record does not assert independent reproduction. ### MermaidFlow Hotpot Score: 32.67. Result ID: 6a1ec734fbc373592c1a4aad. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"Qwen3-8B"}. This record does not assert independent reproduction. ### MaAS Hotpot Score: 32. Result ID: 6a1ec734fbc373592c1a4aac. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"Qwen3-8B"}. This record does not assert independent reproduction. ### MermaidFlow Hotpot Score: 31.33. Result ID: 6a1ec734fbc373592c1a4aa4. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"Qwen3-4B"}. This record does not assert independent reproduction. ### ADAS Hotpot Score: 31.33. Result ID: 6a1ec734fbc373592c1a4aa9. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"Qwen3-8B"}. This record does not assert independent reproduction. ### AFlow Hotpot Score: 31.33. Result ID: 6a1ec734fbc373592c1a4aab. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"Qwen3-8B"}. This record does not assert independent reproduction. ### AFlow Hotpot Score: 30.67. Result ID: 6a1ec734fbc373592c1a4aa2. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"Qwen3-4B"}. This record does not assert independent reproduction. ### MaAS Hotpot Score: 30.67. Result ID: 6a1ec734fbc373592c1a4aa3. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"Qwen3-4B"}. This record does not assert independent reproduction. ### AgentSquare Hotpot Score: 29.33. Result ID: 6a1ec734fbc373592c1a4aa1. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"Qwen3-4B"}. This record does not assert independent reproduction. ### AgentSquare Hotpot Score: 29.33. Result ID: 6a1ec734fbc373592c1a4aaa. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"Qwen3-8B"}. This record does not assert independent reproduction. ### ADAS Hotpot Score: 28.67. Result ID: 6a1ec734fbc373592c1a4aa0. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"Qwen3-4B"}. This record does not assert independent reproduction. Page 1 of 1. This is an index page, not the full collection.