Causal Reasoning on XCOPA (ZH, ID, SW, Avg)
99Accuracy (ZH)ORACLE
Evaluation results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| ORACLE2026.04 | 99 | 98.6 | 97 | 98.2 | — | |
| ORACLEStrategy=Upper bound selection, Backbone=DeepSeek-v3.12026.04 | 99 | 98.6 | 97 | 98.2 | — | |
| ORACLEBase Model=Llama3.3-70B2026.04 | 99 | 98 | 94.4 | 97.1 | — | |
| NATIVEBase Model=Llama3.3-70B2026.04 | 97.8 | 96.6 | 83 | 92.5 | — | |
| MLP (Ours)Base Model=Llama3.3-70B2026.04 | 97.8 | 97 | 85.8 | 93.5 | — | |
| XGBOOST (Ours)Base Model=Llama3.3-70B2026.04 | 97.6 | 97 | 85.8 | 93.5 | — | |
| TRANSLATE2026.04 | 97.4 | 96.8 | 91.8 | 95.3 | — | |
| XGBOOST2026.04 | 97.4 | 96.6 | 93 | 95.7 | — | |
| MLP2026.04 | 97.4 | 97 | 89.6 | 94.3 | — | |
| TRANSLATEStrategy=Translate-then-inference, Backbone=DeepSeek-v3.12026.04 | 97.4 | 96.8 | 91.8 | 95.3 | — | |
| CLASSIFIER (Ours)Strategy=Learned Routing (XGBoost), Backbone=DeepSeek-v3.12026.04 | 97.4 | 96.6 | 93 | 95.7 | — | |
| TRANSLATEBase Model=Llama3.3-70B2026.04 | 97.2 | 97.4 | 89 | 94.5 | — | |
| NATIVE2026.04 | 97 | 95.8 | 87.4 | 93.4 | — | |
| NATIVEStrategy=Native inference, Backbone=DeepSeek-v3.12026.04 | 97 | 95.8 | 87.4 | 93.4 | — |