Task Efficient Routing on Software scenario
0.0138CostForce Weak
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| Force WeakModel=qwen2026.01 | 0.0138 | 65.2 | — | |
| Force WeakModel=qwen2026.01 | 0.0138 | 0.652 | 97.8 | |
| CASTERModel=qwen2026.01 | 0.0186 | 53.1 | — | |
| CASTERModel=qwen2026.01 | 0.0186 | 0.531 | 100 | |
| Force StrongModel=qwen2026.01 | 0.0397 | — | — | |
| Force StrongModel=qwen2026.01 | 0.0397 | — | 99.3 | |
| Force WeakModel=openai2026.01 | 0.0498 | 96.6 | — | |
| Force WeakModel=openai2026.01 | 0.0498 | 0.966 | 82.2 | |
| CASTERModel=deepseek2026.01 | 0.1106 | 52.9 | — | |
| CASTERModel=deepseek2026.01 | 0.1106 | 0.529 | 98.1 | |
| Force WeakModel=deepseek2026.01 | 0.1391 | 40.7 | — | |
| Force WeakModel=deepseek2026.01 | 0.1391 | 0.407 | 98.3 | |
| Force StrongModel=deepseek2026.01 | 0.2346 | — | — | |
| Force StrongModel=deepseek2026.01 | 0.2346 | — | 98.6 | |
| CASTERModel=openai2026.01 | 0.4052 | 72.4 | — | |
| CASTERModel=openai2026.01 | 0.4052 | 0.724 | 97 | |
| Force WeakModel=gemini2026.01 | 0.4412 | 70.8 | — | |
| Force WeakModel=gemini2026.01 | 0.4412 | 0.708 | 99.8 | |
| CASTERStrategy=Dynamic Routing2026.01 | 0.58 | 48 | — | |
| CASTERModel=gemini2026.01 | 0.8783 | 41.8 | — | |
| CASTERModel=gemini2026.01 | 0.8783 | 0.418 | 100 | |
| Force WeakModel=claude2026.01 | 0.8858 | 68.6 | — | |
| Force WeakModel=claude2026.01 | 0.8858 | 0.686 | 93.3 | |
| FrugalGPT (Cascade)Strategy=Cascade2026.01 | 1.11 | — | — | |
| CASTERModel=claude2026.01 | 1.3738 | 51.3 | — | |
| CASTERModel=claude2026.01 | 1.3738 | 0.513 | 96.4 | |
| Force StrongModel=openai2026.01 | 1.4658 | — | — | |
| Force StrongModel=openai2026.01 | 1.4658 | — | 95.3 | |
| Force StrongModel=gemini2026.01 | 1.5087 | — | — | |
| Force StrongModel=gemini2026.01 | 1.5087 | — | 100 | |
| Force StrongModel=claude2026.01 | 2.8204 | — | — | |
| Force StrongModel=claude2026.01 | 2.8204 | — | 100 |