Question Answering on HotpotQA (Perf, Cost, Reward)
0.4335PerfRouteLLM
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| RouteLLMalpha=0.8, beta=0.22026.01 | 0.4335 | 0.2111 | 0.3046 | |
| IRT-Routeralpha=0.8, beta=0.22026.01 | 0.4299 | 0.1377 | 0.3164 | |
| HAPSalpha=0.8, beta=0.22026.01 | 0.4194 | 0.0886 | 0.3178 | |
| HAPSalpha=0.2, beta=0.82026.01 | 0.4181 | 0.0861 | 0.0148 | |
| HAPSalpha=0.5, beta=0.52026.01 | 0.3709 | 0.0857 | 0.1426 | |
| All Strongalpha=0.8, beta=0.22026.01 | 0.3697 | 0.3362 | 0.2285 | |
| All Strongalpha=0.5, beta=0.52026.01 | 0.3697 | 0.3362 | 0.0168 | |
| All Strongalpha=0.2, beta=0.82026.01 | 0.3697 | 0.3362 | -0.195 | |
| IRT-Routeralpha=0.5, beta=0.52026.01 | 0.3604 | 0.0921 | 0.1341 | |
| RouteLLMalpha=0.5, beta=0.52026.01 | 0.3558 | 0.0939 | 0.131 | |
| IRT-Routeralpha=0.2, beta=0.82026.01 | 0.3552 | 0.0881 | 0.0005 | |
| All Weakalpha=0.8, beta=0.22026.01 | 0.3458 | 0.0923 | 0.2582 | |
| All Weakalpha=0.5, beta=0.52026.01 | 0.3458 | 0.0923 | 0.1267 | |
| All Weakalpha=0.2, beta=0.82026.01 | 0.3458 | 0.0923 | -0.0047 | |
| RouteLLMalpha=0.2, beta=0.82026.01 | 0.3458 | 0.0923 | -0.0047 | |
| GraphRouteralpha=0.8, beta=0.22026.01 | 0.3458 | 0.0923 | 0.2582 | |
| GraphRouteralpha=0.5, beta=0.52026.01 | 0.3458 | 0.0923 | 0.1267 | |
| GraphRouteralpha=0.2, beta=0.82026.01 | 0.3458 | 0.0923 | -0.0047 | |
| Randomalpha=0.8, beta=0.22026.01 | 0.3433 | 0.216 | 0.2314 | |
| Randomalpha=0.5, beta=0.52026.01 | 0.3433 | 0.216 | 0.0636 | |
| Randomalpha=0.2, beta=0.82026.01 | 0.3433 | 0.216 | -0.1042 |