Deep Research on Rubrics
40ScoreDR Tulu
Evaluation results
| Method | Links | |
|---|---|---|
| DR TuluBase Model=Qwen3-8B, Task Type=Deep research, # Seed Prompts=~9K, Rubric Source=GPT-4.1, Judge=GPT-4.1-mini, Training Recipe=SFT → GRPO (1.9K steps), External Model Dependency=GPT-4.1, GPT-4.1-mini, GPT-52026.05 | 40 | |
| SCOPEBase Model=Qwen3-8B, Task Type=Open-ended, # Seed Prompts=0, Rubric Source=Self-generated, Judge=Self-judge, Training Recipe=GRPO (60 steps), External Model Dependency=None2026.05 | 31.3 | |
| WebExplorerBase Model=Qwen3-8B, Task Type=Short-form QA, # Seed Prompts=~12K, Rubric Source=—, Judge=—, Training Recipe=SFT → GRPO (∼380 steps), External Model Dependency=DeepSeek-V3, Gemini 2.5 Flash2026.05 | 29.2 | |
| WebThinker-R1Base Model=QwQ-32B, Task Type=Short-form QA, # Seed Prompts=~3K, Rubric Source=—, Judge=—, Training Recipe=DPO (2 iters), External Model Dependency=Qwen2.5-72B-Instruct2026.05 | 9.5 |