Multilingual Language Understanding on HellaSwag, ARC-C, XNLI, and MMLU (French Translated Test Set)
57.83HellaSwag AccuracyGenKnowSub
Evaluation results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| GenKnowSubSetting=Fr, Evaluation Protocol=Zero-shot, Base Model=Phi-32025.05 | 57.83 | 42.95 | 53.65 | 36.13 | 47.64 | |
| Phi-3Evaluation Protocol=Zero-shot, Base Model=Phi-32025.05 | 57.67 | 34.56 | 50.75 | 33.33 | 44.08 | |
| Mean NormEvaluation Protocol=Zero-shot, Base Model=Phi-32025.05 | 57.42 | 35.91 | 52.42 | 33.25 | 44.75 | |
| SharedEvaluation Protocol=Zero-shot, Base Model=Phi-32025.05 | 57.08 | 40.27 | 50.17 | 35.33 | 45.71 | |
| GenKnowSubSetting=Avg, Evaluation Protocol=Zero-shot, Base Model=Phi-32025.05 | 57.08 | 42.62 | 52.92 | 35.58 | 47.05 | |
| GenKnowSubSetting=De, Evaluation Protocol=Zero-shot, Base Model=Phi-32025.05 | 56.42 | 42.28 | 46.33 | 35.58 | 45.15 | |
| GenKnowSubSetting=En, Evaluation Protocol=Zero-shot, Base Model=Phi-32025.05 | 56.08 | 41.95 | 50.66 | 36.69 | 46.34 | |
| ArrowEvaluation Protocol=Zero-shot, Base Model=Phi-32025.05 | 55.33 | 41.61 | 44.38 | 34.79 | 44.02 |