Multiple Choice Question Answering on MMLU-Pro (Score, SE, Delta(k=1))
63.8MMLU-Pro ScoreLlama3.1 70b Instruct
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| Llama3.1 70b Instructn=12, 187, Decoding Strategy=Greedy2025.02 | 63.8 | 0.44 | 3.9 | |
| Llama3.1 70b Instructn=12, 187, Decoding Strategy=Random Sample, k=502025.02 | 63.4 | 0.4 | — | |
| Qwen 2.5 7B Instructn=12, 187, Decoding Strategy=Greedy2025.02 | 53.3 | 0.45 | 1.3 | |
| Qwen 2.5 7B Instructn=12, 187, Decoding Strategy=Random Sample, k=502025.02 | 53 | 0.36 | — | |
| Llama 3.1 8b Instructn=12, 187, Decoding Strategy=Greedy2025.02 | 46.2 | 0.45 | 10 | |
| Llama 3.1 8b Instructn=12, 187, Decoding Strategy=Random Sample, k=502025.02 | 46.1 | 0.39 | — | |
| Ministral 8B Instructn=12, 187, Decoding Strategy=Greedy2025.02 | 39.7 | 0.44 | 1.5 | |
| Ministral 8B Instructn=12, 187, Decoding Strategy=Random Sample, k=502025.02 | 36.3 | 0.29 | — |