# Open Deep Research Evaluation on DeepResearchEval Source: [Open Deep Research Evaluation on DeepResearchEval]() Collection index: [llms.txt]() Data updated: 2026-06-04T12:15:24.431Z Evaluation of model performance on the Open Deep Research Evaluation task using the DeepResearchEval dataset, measured by Cove, Insight, IF., Clar, and Meta scores. A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. | Scope | Recorded value | | --- | --- | | Task | Open Deep Research Evaluation | | Dataset (including any stated split) | DeepResearchEval | | Metric | Cove Score | | Unit | SCALAR | | Direction | Higher is better | | Results loaded | 16 of 16 | | Coverage | All results currently returned for this benchmark | Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol. ## Metrics - [Cove Score](): Higher is better; unit: SCALAR. - [Insight Score](): Higher is better; unit: SCALAR. - [IF Score](): Higher is better; unit: SCALAR. - [Clar Score](): Higher is better; unit: SCALAR. - [Meta Score](): Higher is better; unit: SCALAR. ## Recorded results | Method | Cove Score | Result date | Report paper | | --- | --- | --- | --- | | SCORE | 6.5 | Not recorded | 6a2106fac10484bac23f06e3 | | SCORE | 6.3 | Not recorded | 6a2106fac10484bac23f06e3 | | DPO | 6.1 | Not recorded | 6a2106fac10484bac23f06e3 | | SCORE | 5.9 | Not recorded | 6a2106fac10484bac23f06e3 | | DPO | 4.6 | Not recorded | 6a2106fac10484bac23f06e3 | | GRPO | 4.6 | Not recorded | 6a2106fac10484bac23f06e3 | | Qwen2.5-7B-Inst. | 4.5 | Not recorded | 6a2106fac10484bac23f06e3 | | Qwen2.5-7B-Inst. | 4.4 | Not recorded | 6a2106fac10484bac23f06e3 | | GRPO | 4.4 | Not recorded | 6a2106fac10484bac23f06e3 | | DPO | 4.3 | Not recorded | 6a2106fac10484bac23f06e3 | | SCORE | 4.3 | Not recorded | 6a2106fac10484bac23f06e3 | | Llama-3.1-8B-Inst. | 3.9 | Not recorded | 6a2106fac10484bac23f06e3 | | Llama-3.1-8B-Inst. | 3.7 | Not recorded | 6a2106fac10484bac23f06e3 | | GRPO | 2.5 | Not recorded | 6a2106fac10484bac23f06e3 | | DPO | 2.4 | Not recorded | 6a2106fac10484bac23f06e3 | | GRPO | 1.2 | Not recorded | 6a2106fac10484bac23f06e3 | ### SCORE Cove Score: 6.5. Result ID: 6a216c42fbc373592c1e76aa. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Agent":"gpt-researcher (Plan-and-Execute Paradigm)","Base Model":"Qwen2.5-7B-Inst."}. This record does not assert independent reproduction. ### SCORE Cove Score: 6.3. Result ID: 6a216c42fbc373592c1e76a2. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Agent":"open-deep-research (ReAct Paradigm)","Base Model":"Qwen2.5-7B-Inst."}. This record does not assert independent reproduction. ### DPO Cove Score: 6.1. Result ID: 6a216c42fbc373592c1e76a9. [Reporting paper]() Recorded evaluation configuration: {"Agent":"gpt-researcher (Plan-and-Execute Paradigm)","Base Model":"Qwen2.5-7B-Inst."}. This record does not assert independent reproduction. ### SCORE Cove Score: 5.9. Result ID: 6a216c42fbc373592c1e76a6. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Agent":"gpt-researcher (Plan-and-Execute Paradigm)","Base Model":"Llama-3.1-8B-Inst."}. This record does not assert independent reproduction. ### DPO Cove Score: 4.6. Result ID: 6a216c42fbc373592c1e76a1. [Reporting paper]() Recorded evaluation configuration: {"Agent":"open-deep-research (ReAct Paradigm)","Base Model":"Qwen2.5-7B-Inst."}. This record does not assert independent reproduction. ### GRPO Cove Score: 4.6. Result ID: 6a216c42fbc373592c1e76a8. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Agent":"gpt-researcher (Plan-and-Execute Paradigm)","Base Model":"Qwen2.5-7B-Inst."}. This record does not assert independent reproduction. ### Qwen2.5-7B-Inst. Cove Score: 4.5. Result ID: 6a216c42fbc373592c1e76a7. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Agent":"gpt-researcher (Plan-and-Execute Paradigm)","Base Model":"Qwen2.5-7B-Inst."}. This record does not assert independent reproduction. ### Qwen2.5-7B-Inst. Cove Score: 4.4. Result ID: 6a216c42fbc373592c1e769f. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Agent":"open-deep-research (ReAct Paradigm)","Base Model":"Qwen2.5-7B-Inst."}. This record does not assert independent reproduction. ### GRPO Cove Score: 4.4. Result ID: 6a216c42fbc373592c1e76a0. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Agent":"open-deep-research (ReAct Paradigm)","Base Model":"Qwen2.5-7B-Inst."}. This record does not assert independent reproduction. ### DPO Cove Score: 4.3. Result ID: 6a216c42fbc373592c1e769d. [Reporting paper]() Recorded evaluation configuration: {"Agent":"open-deep-research (ReAct Paradigm)","Base Model":"Llama-3.1-8B-Inst."}. This record does not assert independent reproduction. ### SCORE Cove Score: 4.3. Result ID: 6a216c42fbc373592c1e769e. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Agent":"open-deep-research (ReAct Paradigm)","Base Model":"Llama-3.1-8B-Inst."}. This record does not assert independent reproduction. ### Llama-3.1-8B-Inst. Cove Score: 3.9. Result ID: 6a216c42fbc373592c1e76a3. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Agent":"gpt-researcher (Plan-and-Execute Paradigm)","Base Model":"Llama-3.1-8B-Inst."}. This record does not assert independent reproduction. ### Llama-3.1-8B-Inst. Cove Score: 3.7. Result ID: 6a216c42fbc373592c1e769b. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Agent":"open-deep-research (ReAct Paradigm)","Base Model":"Llama-3.1-8B-Inst."}. This record does not assert independent reproduction. ### GRPO Cove Score: 2.5. Result ID: 6a216c42fbc373592c1e76a4. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Agent":"gpt-researcher (Plan-and-Execute Paradigm)","Base Model":"Llama-3.1-8B-Inst."}. This record does not assert independent reproduction. ### DPO Cove Score: 2.4. Result ID: 6a216c42fbc373592c1e76a5. [Reporting paper]() Recorded evaluation configuration: {"Agent":"gpt-researcher (Plan-and-Execute Paradigm)","Base Model":"Llama-3.1-8B-Inst."}. This record does not assert independent reproduction. ### GRPO Cove Score: 1.2. Result ID: 6a216c42fbc373592c1e769c. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Agent":"open-deep-research (ReAct Paradigm)","Base Model":"Llama-3.1-8B-Inst."}. This record does not assert independent reproduction. Page 1 of 1. This is an index page, not the full collection.