Language Model Routing and Orchestration on Blind pool
74.3Average Precision (p@1)Direct Claude Opus
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| Direct Claude OpusFamily=Single model, Role in comparison=frontier direct upper bound2026.05 | 74.3 | 5.583 | 1,847.3 | |
| AGENTORCHESTRAFamily=Workflow/agentic, Role in comparison=heavy multi-agent orchestration2026.05 | 67.2 | 1.2281 | 1,650.8 | |
| AOrchestraFamily=Workflow/agentic, Role in comparison=agentic workflow orchestration2026.05 | 63.97 | 1.0039 | 1,530.3 | |
| xRouterFamily=Single-turn router, Role in comparison=strong one-shot learned router2026.05 | 54.72 | 0.25 | 574.1 | |
| SkillOrchestraFamily=Skill-aware, Role in comparison=skill competence-cost orchestration2026.05 | 52.61 | 0.1879 | 522.1 | |
| MasRouterFamily=MAS routing, Role in comparison=roles/collaboration/model routing2026.05 | 51.71 | 0.3128 | 600 | |
| GraphRouterFamily=Single-turn router, Role in comparison=graph/representation router2026.05 | 51.61 | 0.3045 | 573.5 | |
| ICL-RouterFamily=Single-turn router, Role in comparison=training-free in-context router2026.05 | 49.85 | 0.2111 | 516.1 | |
| PromptLLMFamily=Multi-turn router, Role in comparison=prompted LLM-as-router2026.05 | 49.74 | 0.8959 | 732.7 | |
| ATLAS (cluster)Family=Single-turn router, Role in comparison=cluster/profile router2026.05 | 49.3 | 0.4042 | 650.5 | |
| RouterDCFamily=Single-turn router, Role in comparison=contrastive query-to-model router2026.05 | 47.33 | 0.2155 | 520 | |
| Tool OrchestraFamily=Workflow/agentic, Role in comparison=tool workflow orchestration2026.05 | 45.74 | 0.7933 | 1,778.2 | |
| ColdStartLLMRouterFamily=Single-turn router, Role in comparison=cold-start low-data router2026.05 | 42.41 | 0.226 | 530.3 | |
| AutoMixFamily=Cascade/mix, Role in comparison=adaptive cascade inference2026.05 | 40.86 | 0.2908 | 1,228.2 | |
| KNNMultiRoundRouterFamily=Multi-turn router, Role in comparison=retrieval-based multi-round router2026.05 | 38.89 | 0.2006 | 542.8 | |
| ToolLLMFamily=Tool-use model, Role in comparison=tool-specialized model2026.05 | 32.79 | 0.1359 | 656.6 |