DeepSearch Question Answering on Humanity's Last Exam
14.7NS ScoreAPPO
Evaluation Results
| Method | Links | ||||
|---|---|---|---|---|---|
| APPOBackbone=Qwen3-8B, Method Strategy=APPO, Method Category=RL-based Method2026.06 | 14.7 | 8.3 | 13 | 13.4 | |
| Qwen3-32B-thinkingBackbone=Qwen3-32B, Method Category=Reference2026.06 | 14.6 | 9.8 | 8.4 | 12.6 | |
| o1-previewBackbone=o1-preview, Method Category=Reference2026.06 | 12.9 | 8.1 | 6.6 | 11.1 | |
| QwQ-32BBackbone=QwQ-32B, Method Category=Reference2026.06 | 11.5 | 7.3 | 5.2 | 9.6 | |
| APPOBackbone=Qwen3-14B, Method Strategy=APPO, Method Category=RL-based Method2026.06 | 11.5 | 12.8 | 14.5 | 12.3 | |
| ARPOBackbone=Qwen3-14B, Method Strategy=ARPO, Method Category=RL-based Method2026.06 | 10.3 | 10.7 | 13.7 | 10 | |
| DeepSeek-R1-671BBackbone=DeepSeek-R1-671B, Method Category=Reference2026.06 | 8.5 | 8.1 | 9.3 | 8.6 | |
| GRPOBackbone=Qwen3-8B, Method Strategy=GRPO, Method Category=RL-based Method2026.06 | 7.9 | 4 | 10.5 | 7.8 | |
| GRPOBackbone=Qwen3-14B, Method Strategy=GRPO, Method Category=RL-based Method2026.06 | 7.9 | 6.7 | 12.6 | 8.6 | |
| Search-o1Backbone=Qwen3-8B, Method Strategy=Search-o1, Method Category=Single-Enhanced Method2026.06 | 7.6 | 2.7 | 5.3 | 6.4 | |
| WebThinkerBackbone=Qwen3-8B, Method Strategy=WebThinker, Method Category=Single-Enhanced Method2026.06 | 7.3 | 4 | 6.3 | 6.6 | |
| ARPOBackbone=Qwen3-8B, Method Strategy=ARPO, Method Category=RL-based Method2026.06 | 7.3 | 6.7 | 15.8 | 8.8 | |
| WebThinkerBackbone=Qwen3-14B, Method Strategy=WebThinker, Method Category=Single-Enhanced Method2026.06 | 7 | 4 | 9.5 | 7 | |
| DeepSeek-R1-32BBackbone=DeepSeek-R1-32B, Method Category=Reference2026.06 | 6.6 | 5.1 | 6.5 | 6.4 | |
| Search-o1Backbone=Qwen3-14B, Method Strategy=Search-o1, Method Category=Single-Enhanced Method2026.06 | 6.4 | 4 | 10.5 | 6.8 | |
| ReActBackbone=Qwen3-14B, Method Strategy=ReAct, Method Category=Single-Enhanced Method2026.06 | 5.8 | 5.3 | 10.5 | 6.6 | |
| Vanilla RAGBackbone=Qwen3-14B, Method Strategy=Vanilla RAG, Method Category=Single-Enhanced Method2026.06 | 5.5 | 6.3 | 9.4 | 6 | |
| Qwen3-14BBackbone=Qwen3-14B, Method Strategy=Base, Method Category=RL-based Method2026.06 | 5.5 | 6.7 | 11.6 | 6.8 | |
| Vanilla RAGBackbone=Qwen3-8B, Method Strategy=Vanilla RAG, Method Category=Single-Enhanced Method2026.06 | 5.1 | 1.6 | 12.9 | 5.8 | |
| ReActBackbone=Qwen3-8B, Method Strategy=ReAct, Method Category=Single-Enhanced Method2026.06 | 4.2 | 4 | 6.3 | 4.6 | |
| Qwen3-8BBackbone=Qwen3-8B, Method Strategy=Base, Method Category=RL-based Method2026.06 | 3.9 | 2.7 | 8.4 | 4.6 | |
| GPT-4oBackbone=GPT-4o, Method Category=Reference2026.06 | 2.7 | 1.2 | 3.2 | 2.6 |