LLM Routing on Heterogeneous four-agent system Gemma, Phi, Qwen (test)
94AccuracyOracle
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| OracleScenario=Performance First, alpha=0.22025.10 | 94 | 0.41 | 86 | |
| OracleScenario=Balance, alpha=0.52025.10 | 94 | 0.41 | 74 | |
| OracleScenario=Cost First, alpha=0.82025.10 | 94 | 0.41 | 61 | |
| Largest LLMScenario=Performance First, alpha=0.22025.10 | 85 | 0.9 | 67 | |
| Largest LLMScenario=Balance, alpha=0.52025.10 | 85 | 0.9 | 40 | |
| Largest LLMScenario=Cost First, alpha=0.82025.10 | 85 | 0.9 | 13 | |
| GraphRouterScenario=Performance First, alpha=0.22025.10 | 82 | 0.72 | 68 | |
| DiSRouter (SFT)Scenario=Performance First, alpha=0.22025.10 | 82 | 0.64 | 69 | |
| DiSRouter (+ RL)Scenario=Performance First, alpha=0.22025.10 | 82 | 0.56 | 71 | |
| FrugalGPTScenario=Performance First, alpha=0.22025.10 | 81 | 0.68 | 67 | |
| GraphRouterScenario=Balance, alpha=0.52025.10 | 81 | 0.7 | 46 | |
| DiSRouter (SFT)Scenario=Balance, alpha=0.52025.10 | 80 | 0.59 | 51 | |
| DiSRouter (+ RL)Scenario=Balance, alpha=0.52025.10 | 80 | 0.53 | 54 | |
| DiSRouter (SFT)Scenario=Cost First, alpha=0.82025.10 | 80 | 0.58 | 34 | |
| DiSRouter (+ RL)Scenario=Cost First, alpha=0.82025.10 | 80 | 0.52 | 38 | |
| RouteLLMScenario=Performance First, alpha=0.22025.10 | 78 | 0.64 | 65 | |
| RouteLLMScenario=Balance, alpha=0.52025.10 | 78 | 0.64 | 46 | |
| RouteLLMScenario=Cost First, alpha=0.82025.10 | 78 | 0.64 | 27 | |
| RandomScenario=Performance First, alpha=0.22025.10 | 71 | 0.56 | 60 | |
| RandomScenario=Balance, alpha=0.52025.10 | 71 | 0.56 | 43 | |
| RandomScenario=Cost First, alpha=0.82025.10 | 70 | 0.56 | 25 | |
| FrugalGPTScenario=Balance, alpha=0.52025.10 | 66 | 0.35 | 49 | |
| GraphRouterScenario=Cost First, alpha=0.82025.10 | 51 | 0.26 | 30 | |
| FrugalGPTScenario=Cost First, alpha=0.82025.10 | 49 | 0.19 | 34 | |
| Smallest LLMScenario=Performance First, alpha=0.22025.10 | 41 | 0.15 | 38 | |
| Smallest LLMScenario=Balance, alpha=0.52025.10 | 41 | 0.15 | 34 | |
| Smallest LLMScenario=Cost First, alpha=0.82025.10 | 41 | 0.15 | 29 |