Multitask Language Understanding on MMLU (Per-Language Accuracy Breakdown) benchmark leaderboard · SOTA2 Research

Multitask Language Understanding on MMLU (Per-Language Accuracy Breakdown)

89Arabic Accuracyo1

Evaluation Results

Method	Links
o12024.12		89	87.34	88.92	92.3	89.32	89.04	88.33	88.61	89.7	88.87	88.24	89.52	89.92	85.4	75.38
o1-preview2024.12		88.21	86.22	88	90.8	88.61	85.73	87.82	88.21	88.72	87.88	88.15	88.59	88.93	84.79	73.73
GPT-4o2024.12		81.55	80.07	83.35	88.7	84.37	82.92	80.61	83.44	84.35	82.87	82.62	84.27	84.93	77.08	61.95
o1-mini2024.12		79.45	77.25	81.8	85.2	82.12	81.22	78.87	81.74	82.22	81.29	80.2	82.43	83.03	70.15	58.07
GPT-4o-mini2024.12		70.89	65.77	73.05	82	76.59	74.31	69.16	74.52	76.4	72.55	72.03	76.77	77.37	61.91	45.83