Deep Research Report Generation on DRB
50.7Overall ScoreGPT-5 + Search
Evaluation results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| GPT-5 + SearchBackbone=GPT-52026.06 | 50.7 | 49.7 | 51.5 | 51.6 | 48.5 | |
| Gemini Deep ResearchBackbone=Proprietary2026.06 | 48.8 | 48.5 | 48.5 | 49.2 | 49.4 | |
| OpenAI Deep ResearchBackbone=o3/o42026.06 | 46.9 | 46.8 | 45.2 | 49.2 | 47.1 | |
| Gemini 3 Pro + SearchBackbone=Gemini 3 Pro2026.06 | 46.3 | 43.4 | 44.9 | 49.8 | 49 | |
| DEEPRUBRIC (140-step, ≈750h)Backbone=Qwen3-8B, Steps=140-step, Compute Cost=≈750h2026.06 | 43.6 | 41.7 | 41.8 | 48.4 | 42.5 | |
| DR Tulu-8B (1900-step, ≈9700h)Backbone=Qwen3-8B, Steps=1900-step, Compute Cost=≈9700h2026.06 | 43.4 | 41.7 | 41.8 | 48.2 | 41.3 | |
| Perplexity Deep ResearchBackbone=Proprietary2026.06 | 42.3 | 40.7 | 39.3 | 46.4 | 44.3 | |
| DEEPRUBRIC (75-step, ≈402h)Backbone=Qwen3-8B, Steps=75-step, Compute Cost=≈402h2026.06 | 41.9 | 39.7 | 40 | 46.9 | 41.5 | |
| DR Tulu-8B (1000-step, ≈6000h)Backbone=Qwen3-8B, Steps=1000-step, Compute Cost=≈6000h2026.06 | 41.8 | 39.5 | 39.4 | 47.3 | 41.6 | |
| Perplexity-Sonar (High)Backbone=Proprietary2026.06 | 40.7 | 37.4 | 36.1 | 45.7 | 44.7 | |
| Tongyi DeepResearch-30B-A3BBackbone=Qwen2.5-30B2026.06 | 40.6 | 39.1 | 34.3 | 46.8 | 45.4 | |
| WebThinker-32B-DPO (report)Backbone=Qwen2.5-32B (report)2026.06 | 40.6 | 39.4 | 35.4 | 46 | 43.5 | |
| QwQ-32B + RAGBackbone=QwQ-32B2026.06 | 40.3 | 38.1 | 34.8 | 47 | 44.6 | |
| DR Tulu-8B (SFT)Backbone=Qwen3-8B2026.06 | 39 | 36.3 | 35.3 | 45.5 | 39.5 | |
| DEEPRUBRIC (SFT)Backbone=Qwen3-8B, Stage=SFT2026.06 | 38 | 33.8 | 36 | 43.5 | 40.1 | |
| WebThinker (report)Backbone=QwQ-32B (report)2026.06 | 37.9 | 36.2 | 32.6 | 43.2 | 42.9 | |
| WebExplorer-8BBackbone=Qwen2.5-8B2026.06 | 36.7 | 33.7 | 28.5 | 45.7 | 42.2 | |
| Ai2 ScholarQABackbone=Claude Sonnet2026.06 | 36.1 | 35.1 | 32 | 40.5 | 38.9 | |
| Claude-Sonnet SearchBackbone=Claude Sonnet2026.06 | 34.5 | 39 | 37.7 | 45.8 | 41.5 | |
| Qwen3-8B + RAGBackbone=Qwen3-8B2026.06 | 33.3 | 29.4 | 27 | 40.2 | 41.1 | |
| WebThinker-32B-DPOBackbone=Qwen2.5-32B2026.06 | 23.3 | 19.7 | 12.3 | 36.8 | 26.3 | |
| Qwen3-8B + SearchBackbone=Qwen3-8B2026.06 | 18.2 | 14.3 | 8.7 | 29.5 | 24.4 | |
| Search-R1-7BBackbone=Qwen2.5-7B2026.06 | 9.5 | 5.2 | 2.1 | 18.6 | 16.8 | |
| ASearcher-Web-7BBackbone=Qwen2.5-7B2026.06 | 7.8 | 5.1 | 1.7 | 15.2 | 11.8 |