Code Generation on Code/SE Macro-aggregate
77.8Pass@1Uno-Orchestra
Evaluation Results
| Method | Links | ||
|---|---|---|---|
| Uno-Orchestratok=411, USD/q=0.10112026.05 | 77.8 | 82.2 | |
| Uno-tree-GRPOtok=419, USD/q=0.17122026.05 | 76.9 | 81.7 | |
| Uno-GRPOtok=391, USD/q=0.16102026.05 | 75.4 | 80.5 | |
| AgentOrchestratok=1724, USD/q=1.21182026.05 | 71.4 | 76.5 | |
| AOrchestratok=1600, USD/q=0.99322026.05 | 63.5 | 72.4 | |
| Tool Orchestratok=1896, USD/q=0.81002026.05 | 57.9 | 62.7 | |
| xRoutertok=584, USD/q=0.24902026.05 | 56.1 | 62.2 | |
| MasRoutertok=610, USD/q=0.31212026.05 | 56.1 | 61.6 | |
| SkillOrchestratok=533, USD/q=0.18702026.05 | 55.2 | 61.2 | |
| ICL-Routertok=525, USD/q=0.21002026.05 | 53.2 | 59.2 | |
| GraphRoutertok=585, USD/q=0.30842026.05 | 52 | 58.9 | |
| ATLAS (RL)tok=736, USD/q=0.50322026.05 | 51.4 | 58.3 | |
| Puppeteertok=694, USD/q=0.46262026.05 | 51.3 | 58.3 | |
| PromptLLMtok=747, USD/q=0.90712026.05 | 51.1 | 58.3 | |
| R2-Reasonertok=601, USD/q=0.35642026.05 | 50.8 | 57.8 | |
| RouterDCtok=529, USD/q=0.21462026.05 | 50.6 | 56.5 | |
| ATLAS (cluster)tok=662, USD/q=0.40862026.05 | 49.8 | 56.9 | |
| AutoMixtok=1250, USD/q=0.29552026.05 | 46.9 | 55 | |
| WideSeek-R1tok=860, USD/q=0.64852026.05 | 46.7 | 54.1 | |
| ColdStart-LLMtok=540, USD/q=0.22882026.05 | 43.9 | 51.5 | |
| ToolLLMtok=667, USD/q=0.13552026.05 | 26.3 | 33.1 | |
| Router-R1tok=380, USD/q=0.00052026.05 | 12.1 | 19.4 |