General Question Answering on NQ (Exact Match)
54.8Exact Match (EM)SkillOrchestra+
Evaluation results
| Method | Links | |
|---|---|---|
| SkillOrchestra+Routing Strategy=Skill-based Routing, Orchestrator Configuration=Best/Multiple2026.02 | 54.8 | |
| SkillOrchestraRouting Strategy=Skill-based Routing, Orchestrator Configuration=Single2026.02 | 54.2 | |
| π-PlayBase Model=Qwen3-8B2026.04 | 43 | |
| Agentic-RSearch Agent=Our Search Agent, Search Agent Domain=in-domain2026.01 | 42.43 | |
| E5Search Agent=Our Search Agent, Search Agent Domain=in-domain2026.01 | 42.18 | |
| Dr.ZeroBase Model=Qwen3-8B2026.04 | 42.1 | |
| π-PlayBase Model=Qwen3-4B-Instruct-25072026.04 | 41.8 | |
| REPLUGSearch Agent=Our Search Agent, Search Agent Domain=in-domain2026.01 | 41.46 | |
| LLM-EmbedderSearch Agent=Our Search Agent, Search Agent Domain=in-domain2026.01 | 41.32 | |
| Dr.ZeroBase Model=Qwen3-4B-Instruct-25072026.04 | 41.3 | |
| REPLUGSearch Agent=R1-Searcher, Search Agent Domain=out-of-domain2026.01 | 40.94 | |
| SCARLetSearch Agent=Our Search Agent, Search Agent Domain=in-domain2026.01 | 40.6 | |
| Dr.ZeroBase Model=Qwen3-4B2026.04 | 40.4 | |
| BGESearch Agent=Our Search Agent, Search Agent Domain=in-domain2026.01 | 40.36 | |
| SQLM*Base Model=Qwen3-4B2026.04 | 39.9 | |
| Agentic-RSearch Agent=R1-Searcher, Search Agent Domain=out-of-domain2026.01 | 39.63 | |
| Search-R1Base Model=Qwen3-4B-Instruct-25072026.04 | 39.5 | |
| E5Search Agent=R1-Searcher, Search Agent Domain=out-of-domain2026.01 | 39.39 | |
| SQLM*Base Model=Qwen3-4B-Instruct-25072026.04 | 39 | |
| SQLM*Base Model=Qwen3-8B2026.04 | 38.9 | |
| LLM-EmbedderSearch Agent=R1-Searcher, Search Agent Domain=out-of-domain2026.01 | 38.8 | |
| Router-R1Routing Strategy=RL-based Routing2026.02 | 38.8 | |
| π-PlayBase Model=Qwen3-4B2026.04 | 38.1 | |
| BGESearch Agent=R1-Searcher, Search Agent Domain=out-of-domain2026.01 | 36.34 | |
| SCARLetSearch Agent=R1-Searcher, Search Agent Domain=out-of-domain2026.01 | 36.34 | |
| Search-R1Base Model=Qwen3-8B2026.04 | 35.7 | |
| Search-R1Base Model=Qwen3-4B2026.04 | 35.2 | |
| Agentic-RSearch Agent=SimpleDeepSearcher, Search Agent Domain=out-of-domain2026.01 | 34.62 | |
| REPLUGSearch Agent=SimpleDeepSearcher, Search Agent Domain=out-of-domain2026.01 | 33.71 | |
| E5Search Agent=SimpleDeepSearcher, Search Agent Domain=out-of-domain2026.01 | 33.65 | |
| LLM-EmbedderSearch Agent=SimpleDeepSearcher, Search Agent Domain=out-of-domain2026.01 | 33.62 | |
| BGESearch Agent=SimpleDeepSearcher, Search Agent Domain=out-of-domain2026.01 | 32.82 | |
| Search-R1Routing Strategy=No Routing2026.02 | 32.8 | |
| SCARLetSearch Agent=SimpleDeepSearcher, Search Agent Domain=out-of-domain2026.01 | 32.38 | |
| ToolForgeBase Model=Qwen3-8B2026.04 | 30.3 | |
| Prompt LLMRouting Strategy=Heuristic & Discriminative Routing2026.02 | 30 | |
| RAGRouting Strategy=No Routing2026.02 | 29.8 | |
| Largest LLMRouting Strategy=Heuristic & Discriminative Routing2026.02 | 29.6 | |
| RouterDCRouting Strategy=Heuristic & Discriminative Routing2026.02 | 27.8 | |
| GraphRouterRouting Strategy=Heuristic & Discriminative Routing2026.02 | 27.6 | |
| FrugalGPTRouting Strategy=Heuristic & Discriminative Routing2026.02 | 26.5 | |
| KNN RouterRouting Strategy=Heuristic & Discriminative Routing2026.02 | 26.2 | |
| Prompt LLM+Routing Strategy=Heuristic & Discriminative Routing, Turn Protocol=multi turn2026.02 | 25.8 | |
| MLP RouterRouting Strategy=Heuristic & Discriminative Routing2026.02 | 25.2 | |
| KNN Router+Routing Strategy=Heuristic & Discriminative Routing, Turn Protocol=multi turn2026.02 | 23.6 | |
| BERT RouterRouting Strategy=Heuristic & Discriminative Routing2026.02 | 23 | |
| SFTRouting Strategy=No Routing2026.02 | 21.2 | |
| ReActBase Model=Qwen3-4B-Instruct-25072026.04 | 18.4 | |
| ReActBase Model=Qwen3-8B2026.04 | 15.9 | |
| CoTRouting Strategy=No Routing2026.02 | 12.6 | |
| VanillaRouting Strategy=No Routing2026.02 | 9.2 | |
| ReActBase Model=Qwen3-4B2026.04 | 7.2 |