# Report Generation on DeepResearch Gym (test) Source: [Report Generation on DeepResearch Gym (test)]() Collection index: [llms.txt]() Data updated: 2026-06-19T08:23:38.291Z Evaluation of model performance on generating reports using the test split of the DeepResearch Gym dataset, focusing on metrics related to report quality such as Clarity, Depth, Balance, Breadth, Support, and Insight. A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. | Scope | Recorded value | | --- | --- | | Task | Report Generation | | Dataset (including any stated split) | DeepResearch Gym (test) | | Metric | Clarity | | Unit | PERCENT | | Direction | Higher is better | | Results loaded | 10 of 10 | | Coverage | All results currently returned for this benchmark | Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol. ## Metrics - [Clarity](): Higher is better; unit: PERCENT. - [Depth](): Higher is better; unit: PERCENT. - [Balance](): Higher is better; unit: PERCENT. - [Breadth](): Higher is better; unit: PERCENT. - [Support](): Higher is better; unit: PERCENT. - [Insight](): Higher is better; unit: PERCENT. - [Average Score](): Higher is better; unit: PERCENT. ## Recorded results | Method | Clarity | Result date | Report paper | | --- | --- | --- | --- | | Prompt | 70.4 | Not recorded | 6a34cd4b206542251d7c6185 | | RAG | 64.6 | Not recorded | 6a34cd4b206542251d7c6185 | | SCAFFOLDAGENT | 63.3 | Not recorded | 6a34cd4b206542251d7c6185 | | IRCoT | 62.3 | Not recorded | 6a34cd4b206542251d7c6185 | | WebWeaver | 62.1 | Not recorded | 6a34cd4b206542251d7c6185 | | StackPlanner | 62.1 | Not recorded | 6a34cd4b206542251d7c6185 | | ReAct | 60.5 | Not recorded | 6a34cd4b206542251d7c6185 | | EDR | 55.8 | Not recorded | 6a34cd4b206542251d7c6185 | | WebShaper | 49.4 | Not recorded | 6a34cd4b206542251d7c6185 | | STORM | 31 | Not recorded | 6a34cd4b206542251d7c6185 | ### Prompt Clarity: 70.4. Result ID: 6a34fc0efbc373592c324a4c. [Reporting paper]() Recorded evaluation configuration: {"Paradigm":"Naive","Backbone":"Qwen3-32B"}. This record does not assert independent reproduction. ### RAG Clarity: 64.6. Result ID: 6a34fc0efbc373592c324a4d. [Reporting paper]() Recorded evaluation configuration: {"Paradigm":"Naive","Backbone":"Qwen3-32B"}. This record does not assert independent reproduction. ### SCAFFOLDAGENT Clarity: 63.3. Result ID: 6a34fc0efbc373592c324a55. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Paradigm":"Multi-Agent","Backbone":"Qwen3-32B"}. This record does not assert independent reproduction. ### IRCoT Clarity: 62.3. Result ID: 6a34fc0efbc373592c324a4f. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Paradigm":"Single-Agent","Backbone":"Qwen3-32B"}. This record does not assert independent reproduction. ### WebWeaver Clarity: 62.1. Result ID: 6a34fc0efbc373592c324a52. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Paradigm":"Multi-Agent","Backbone":"Qwen3-32B"}. This record does not assert independent reproduction. ### StackPlanner Clarity: 62.1. Result ID: 6a34fc0efbc373592c324a54. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Paradigm":"Multi-Agent","Backbone":"Qwen3-32B"}. This record does not assert independent reproduction. ### ReAct Clarity: 60.5. Result ID: 6a34fc0efbc373592c324a4e. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Paradigm":"Single-Agent","Backbone":"Qwen3-32B"}. This record does not assert independent reproduction. ### EDR Clarity: 55.8. Result ID: 6a34fc0efbc373592c324a53. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Paradigm":"Multi-Agent","Backbone":"Qwen3-32B"}. This record does not assert independent reproduction. ### WebShaper Clarity: 49.4. Result ID: 6a34fc0efbc373592c324a50. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Paradigm":"Single-Agent","Backbone":"Qwen3-32B"}. This record does not assert independent reproduction. ### STORM Clarity: 31. Result ID: 6a34fc0efbc373592c324a51. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Paradigm":"Multi-Agent","Backbone":"Qwen3-32B"}. This record does not assert independent reproduction. Page 1 of 1. This is an index page, not the full collection.