Question Answering on MMLU-Pro (Subject-Specific ACC/LEN)
25.7Law AccuracyMERA
Evaluation results
| Method | Links | ||||||
|---|---|---|---|---|---|---|---|
| MERABase Model=DeepSeek-R1-Distill-Qwen-7B2025.08 | 25.7 | 814 | 28.5 | 4,092 | 36.9 | 3,854 | |
| FCS+Ref.Base Model=DeepSeek-R1-Distill-Qwen-7B2025.08 | 24.1 | 973 | 27.9 | 5,518 | 34.9 | 5,098 | |
| OriginalBase Model=DeepSeek-R1-Distill-Qwen-7B2025.08 | 23.5 | 1,528 | 26.4 | 5,412 | 34.1 | 4,978 | |
| O1-PrunerBase Model=DeepSeek-R1-Distill-Qwen-7B2025.08 | 22.8 | 1,054 | 25.6 | 4,981 | 35.6 | 4,587 | |
| DASTBase Model=DeepSeek-R1-Distill-Qwen-7B2025.08 | 21.4 | 1,249 | 27.1 | 5,821 | 33.5 | 5,243 |