Subject Knowledge Evaluation on MMLU
48.7MMLU Overall AccuracyPractical (R̄_FPO)
Evaluation results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| Practical (R̄_FPO)Evaluation Protocol=50-question stratified sample per subject2026.05 | 48.7 | 41.2 | 54 | 53.1 | 50.6 | |
| Relaxed (R̃_FPO)Evaluation Protocol=50-question stratified sample per subject2026.05 | 48.6 | 41.5 | 54.3 | 52.6 | 49.5 | |
| Standard RLHFEvaluation Protocol=50-question stratified sample per subject2026.05 | 48.4 | 41.3 | 53.7 | 52.5 | 50 |