Mathematical Reasoning on AIME (pass@1)
66.5Pass@1Uno-Orchestra
Evaluation Results
| Method | Links | |
|---|---|---|
| Uno-Orchestra2026.05 | 66.5 | |
| AOrchestra2026.05 | 36.9 | |
| xRouter2026.05 | 34.9 | |
| AgentOrchestra2026.05 | 31.2 | |
| ColdStart-LLM2026.05 | 16.4 | |
| SECBackbone=Qwen3-4B-Base, Seeds=12026.06 | 12.9 | |
| TACBackbone=Qwen3-4B-Base, Seeds=12026.06 | 12.3 | |
| RandomBackbone=Qwen3-4B-Base, Seeds=12026.06 | 10.8 | |
| VeriGatePolicy Model=Qwen2.5-7B-Instruct, PRM Model=Qwen-PRM-7B2026.05 | 10 | |
| GRPO-VanillaPolicy Model=Qwen2.5-7B-Instruct, PRM Model=Qwen-PRM-7B2026.05 | 6.67 | |
| SECBackbone=Qwen3-1.7B2026.06 | 5.4 | |
| RandomBackbone=Qwen3-1.7B2026.06 | 5.3 | |
| M2OBackbone=Qwen3-1.7B2026.06 | 5.2 | |
| TACBackbone=Llama3.2-3B2026.06 | 5.2 | |
| TACBackbone=Qwen3-1.7B2026.06 | 5.1 | |
| M2OBackbone=Llama3.2-3B2026.06 | 4.9 | |
| BaseBackbone=Qwen3-1.7B2026.06 | 4.3 | |
| SECBackbone=Llama3.2-3B2026.06 | 3.6 | |
| RandomBackbone=Llama3.2-3B2026.06 | 3.4 | |
| VeriGate (-Gate)Policy Model=Qwen2.5-1.5B-Instruct, PRM Model=Qwen-PRM-7B2026.05 | 3.33 | |
| PRM-as-ORMPolicy Model=Qwen2.5-7B-Instruct, PRM Model=Qwen-PRM-7B2026.05 | 3.33 | |
| Dr.GRPOPolicy Model=Qwen2.5-1.5B-Instruct, PRM Model=Qwen-PRM-7B2026.05 | 3.3 | |
| BaseBackbone=Llama3.2-3B2026.06 | 2.2 | |
| TACBackbone=Qwen3-0.6B-Base, Seeds=12026.06 | 1.7 | |
| RandomBackbone=Qwen3-0.6B-Base, Seeds=12026.06 | 1.5 | |
| SECBackbone=Qwen3-0.6B-Base, Seeds=12026.06 | 0.9 | |
| GRPO-VanillaPolicy Model=Qwen2.5-1.5B-Instruct, PRM Model=Qwen-PRM-7B2026.05 | 0 | |
| DAPOPolicy Model=Qwen2.5-1.5B-Instruct, PRM Model=Qwen-PRM-7B2026.05 | 0 | |
| PRM-as-ORMPolicy Model=Qwen2.5-1.5B-Instruct, PRM Model=Qwen-PRM-7B2026.05 | 0 | |
| PRM-as-ORM (+Gate)Policy Model=Qwen2.5-1.5B-Instruct, PRM Model=Qwen-PRM-7B2026.05 | 0 | |
| VeriGate (Simplified)Policy Model=Qwen2.5-1.5B-Instruct, PRM Model=Qwen-PRM-7B2026.05 | 0 | |
| VeriGatePolicy Model=Qwen2.5-1.5B-Instruct, PRM Model=Qwen-PRM-7B2026.05 | 0 |