Deep search on HLE
11.4AccuracyECHO
Evaluation results
| Method | Links | ||
|---|---|---|---|
| ECHOBackbone=Qwen3-32B-Instruct2026.06 | 11.4 | — | |
| ECHO w/ Top-K retrievalBackbone=Qwen3-32B-Instruct2026.06 | 10.6 | — | |
| ECHO w/ Top-K retrieval & w/o turn summaryBackbone=Qwen3-32B-Instruct2026.06 | 10 | — | |
| ECHO w/o Traceable CA & Turn-level ISBackbone=Qwen3-32B-Instruct2026.06 | 9.4 | — | |
| SUPOBackbone=Qwen3-32B-Instruct2026.06 | 9.2 | — | |
| ECHOBackbone=Qwen3-30B-A3B-Instruct2026.06 | 9.2 | — | |
| GRPOBackbone=Qwen3-32B-Instruct2026.06 | 8.8 | — | |
| ECHO w/o traceable CABackbone=Qwen3-32B-Instruct2026.06 | 8.8 | — | |
| MusiqueBackbone=QWEN-7B, Training data=Musique, Search Engine=Google Search, Max queries=202026.01 | 8 | — | |
| SUPOBackbone=Qwen3-30B-A3B-Instruct2026.06 | 8 | — | |
| GRPOBackbone=Qwen3-30B-A3B-Instruct2026.06 | 7.8 | — | |
| SAGEBackbone=QWEN-7B, Training data=SAGE (Ours), Search Engine=Google Search, Max queries=202026.01 | 7 | — | |
| SAGEBackbone=QWEN-3B, Training data=SAGE (Ours), Search Engine=Google Search, Max queries=202026.01 | 5.5 | — | |
| NQ + HotpotQABackbone=QWEN-3B, Training data=NQ + HotpotQA, Search Engine=Google Search, Max queries=202026.01 | 5 | — | |
| NQ + HotpotQABackbone=QWEN-7B, Training data=NQ + HotpotQA, Search Engine=Google Search, Max queries=202026.01 | 4.5 | — | |
| MusiqueBackbone=QWEN-3B, Training data=Musique, Search Engine=Google Search, Max queries=202026.01 | 4 | — |