Multiple-choice Question Answering on Global-MMLU (Multilingual Evaluation)
90.5Accuracy (ZH)ORACLE
Evaluation results
| Method | Links | |||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| ORACLE2026.04 | 90.5 | 90.6 | 91.2 | 89.1 | 88.4 | 90.8 | 89.5 | 86.5 | 85.3 | 72.8 | 88.3 | |
| ORACLEStrategy=Upper bound selection, Backbone=DeepSeek-v3.12026.04 | 90.5 | 90.6 | 91.2 | 89.1 | 88.4 | 90.8 | 89.5 | 86.5 | 85.3 | 72.8 | 88.3 | |
| XGBOOST2026.04 | 86.8 | 87.3 | 86 | 84.7 | 83.3 | 86.4 | 84.9 | 80 | 79.8 | 63.7 | 82.3 | |
| CLASSIFIER (Ours)Strategy=Learned Routing (XGBoost), Backbone=DeepSeek-v3.12026.04 | 86.8 | 87.3 | 86 | 84.7 | 83.3 | 86.4 | 84.9 | 80 | 79.8 | 63.7 | 82.3 | |
| NATIVE2026.04 | 86.4 | 86.4 | 84.6 | 83.1 | 79.9 | 85.3 | 36.1 | 72 | 71.7 | 39 | 72.5 | |
| MLP2026.04 | 86.4 | 86.6 | 85.1 | 84.1 | 81.6 | 85.6 | 76.9 | 78.1 | 77.1 | 61.4 | 80.3 | |
| NATIVEStrategy=Native inference, Backbone=DeepSeek-v3.12026.04 | 86.4 | 86.4 | 84.6 | 83.1 | 79.9 | 85.3 | 36.1 | 72 | 71.7 | 39 | 72.5 | |
| TRANSLATE2026.04 | 86 | 86.8 | 85.5 | 84.4 | 83 | 86 | 84.8 | 80 | 79.2 | 61.6 | 81.7 | |
| TRANSLATEStrategy=Translate-then-inference, Backbone=DeepSeek-v3.12026.04 | 86 | 86.8 | 85.5 | 84.4 | 83 | 86 | 84.8 | 80 | 79.2 | 61.6 | 81.7 | |
| ORACLEBase Model=Llama3.3-70B2026.04 | 85.3 | 87.2 | 85.7 | 83.4 | 81.3 | 85.3 | 82.5 | 76 | 79.3 | 55.1 | 80.1 | |
| XGBOOST (Ours)Base Model=Llama3.3-70B2026.04 | 80 | 82.9 | 79.5 | 76.7 | 73.4 | 80.9 | 76.5 | 68.1 | 69.7 | 43 | 73.1 | |
| MLP (Ours)Base Model=Llama3.3-70B2026.04 | 79.8 | 82.5 | 79.7 | 76.8 | 73.2 | 80.2 | 69.8 | 69.1 | 70.3 | 40.7 | 72.2 | |
| NATIVEBase Model=Llama3.3-70B2026.04 | 79.5 | 82.4 | 79.7 | 74.6 | 69.8 | 79.6 | 60.5 | 54.4 | 67.1 | 23.9 | 67.2 | |
| TRANSLATEBase Model=Llama3.3-70B2026.04 | 78.5 | 80.6 | 76.9 | 76.2 | 72.9 | 78.6 | 76.5 | 67 | 69.8 | 43 | 72 |