Multilingual Language Understanding on German HellaSwag, ARC-Challenge, XNLI, and MMLU (test)
52.48HellaSwagPhi-3
Evaluation results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| Phi-3Evaluation Protocol=Zero-shot, Base Model=Phi-32025.05 | 52.48 | 36.24 | 36.02 | 33.82 | 39.64 | |
| GenKnowSubSetting=En, Evaluation Protocol=Zero-shot, Base Model=Phi-32025.05 | 51.16 | 40.6 | 50.14 | 36.85 | 44.69 | |
| GenKnowSubSetting=Avg, Evaluation Protocol=Zero-shot, Base Model=Phi-32025.05 | 51.08 | 42.62 | 52.33 | 37.92 | 45.99 | |
| Mean NormEvaluation Protocol=Zero-shot, Base Model=Phi-32025.05 | 51 | 36.91 | 33.67 | 33.5 | 38.77 | |
| GenKnowSubSetting=De, Evaluation Protocol=Zero-shot, Base Model=Phi-32025.05 | 50.58 | 42.95 | 50.42 | 37 | 45.24 | |
| GenKnowSubSetting=Fr, Evaluation Protocol=Zero-shot, Base Model=Phi-32025.05 | 50.58 | 42.62 | 49.17 | 37.17 | 44.88 | |
| SharedEvaluation Protocol=Zero-shot, Base Model=Phi-32025.05 | 49.75 | 38.93 | 43 | 36 | 41.92 | |
| ArrowEvaluation Protocol=Zero-shot, Base Model=Phi-32025.05 | 48.58 | 40.94 | 43.45 | 35.4 | 42.09 |