Multi-task Language Understanding on MMLU Pro (Accuracy and #Calls)
62.1AccuracySC
Evaluation results
| Method | Links | ||
|---|---|---|---|
| SCBudget (B)=162025.11 | 62.1 | 16 | |
| ESCBudget (B)=16, Window size (w)=82025.11 | 62.1 | 12.87 | |
| ESCBudget (B)=16, Window size (w)=42025.11 | 62 | 8.99 | |
| ASC-βBudget (B)=162025.11 | 62 | 8.55 | |
| ASC-DirBudget (B)=162025.11 | 62 | 9.07 | |
| CGES-MARSBudget (B)=162025.11 | 62 | 8.8 | |
| CGES-LNS (arith)Budget (B)=16, Aggregation function=arith2025.11 | 61.9 | 8.24 | |
| CGES-LNS (geom)Budget (B)=16, Aggregation function=geom2025.11 | 61.9 | 8.95 | |
| CGES-DeepConf (B10)Budget (B)=16, DeepConf variant=B102025.11 | 61.9 | 8.26 | |
| CGES-DeepConf (tail)Budget (B)=16, DeepConf variant=tail2025.11 | 61.9 | 8.23 | |
| DSCBudget (B)=162025.11 | 60.9 | 5.18 | |
| CGES-RMBudget (B)=162025.11 | 58.3 | 12.31 |