Multiple-choice Question Answering on MMLU-ProX
82.6Average AccuracyORACLE
Evaluation results
| Method | Links | ||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| ORACLEStrategy=Upper bound selection, Backbone=DeepSeek-v3.12026.04 | 82.6 | 85.4 | 85.5 | 85 | 84.2 | 83.3 | 85.5 | 82.8 | 80.6 | 70.8 | — | — | |
| CLASSIFIER (Ours)Strategy=Learned Routing (XGBoost), Backbone=DeepSeek-v3.12026.04 | 77.9 | 80.8 | 81.3 | 80.5 | 79.2 | 78.7 | 80.7 | 79.5 | 76 | 64.6 | — | — | |
| TRANSLATEStrategy=Translate-then-inference, Backbone=DeepSeek-v3.12026.04 | 77.7 | 80.3 | 81 | 80.3 | 79 | 78.5 | 80.5 | 79.5 | 75.6 | 64.5 | — | — | |
| NATIVEStrategy=Native inference, Backbone=DeepSeek-v3.12026.04 | 69.2 | 80.5 | 80.8 | 79.7 | 77.9 | 75.4 | 80.1 | 35.7 | 69.3 | 43.4 | — | — | |
| ORACLEBase Model=Llama3.3-70B2026.04 | 65.4 | 71.3 | 72.3 | 71.4 | 67.3 | 64.4 | 72 | 63.9 | 62.7 | 43.6 | — | — | |
| XGBOOST (Ours)Base Model=Llama3.3-70B2026.04 | 58.5 | 64 | 67.1 | 66.1 | 61 | 55.7 | 66.6 | 55.9 | 54.2 | 36 | — | — | |
| MLP (Ours)Base Model=Llama3.3-70B2026.04 | 58.3 | 64.3 | 67 | 66.2 | 61 | 56.5 | 66.6 | 51.3 | 56.1 | 35.8 | — | — | |
| TRANSLATEBase Model=Llama3.3-70B2026.04 | 53.9 | 57.8 | 59.9 | 58.7 | 54.5 | 52.9 | 59.6 | 56 | 49.9 | 36.2 | — | — | |
| NATIVEBase Model=Llama3.3-70B2026.04 | 53.6 | 63.8 | 66 | 65.4 | 57.8 | 53.4 | 65.5 | 40.2 | 51.6 | 18.8 | — | — | |
| Naive Fine-tuningBackbone=LLaMA-3.1-Instruct-8B, Number of random seeds=32026.04 | 25.3 | — | — | — | — | — | — | — | — | — | — | — | |
| COMPASS-ECDABackbone=LLaMA-3.1-Instruct-8B, Number of random seeds=32026.04 | 25.2 | — | — | — | — | — | — | — | — | — | — | — | |
| Full RetrainingBackbone=LLaMA-3.1-Instruct-8B, Number of random seeds=32026.04 | 24.6 | — | — | — | — | — | — | — | — | — | — | — | |
| EWCBackbone=LLaMA-3.1-Instruct-8B, Number of random seeds=32026.04 | 24.5 | — | — | — | — | — | — | — | — | — | — | — | |
| Random RehearsalBackbone=LLaMA-3.1-Instruct-8B, Number of random seeds=32026.04 | 23.9 | — | — | — | — | — | — | — | — | — | — | — |