Question Answering on MMLU-Redux
92.91AccuracyDeepSeek-R1
Evaluation results
| Method | Links | |
|---|---|---|
| DeepSeek-R1Model Size=671B, Mode=Thinking2026.02 | 92.91 | |
| GPT-4.1Mode=No-Thinking2026.02 | 92.43 | |
| Qwen3Model Size=32B, Mode=Thinking2026.02 | 90.92 | |
| GPT-4o + QuaSARBackbone=GPT-4o, Prompting Strategy=QuaSAR2025.02 | 90.2 | |
| Qwen3Model Size=14B, Mode=Thinking2026.02 | 88.61 | |
| GPT-4o + CoMATBackbone=GPT-4o, Prompting Strategy=CoMAT2025.02 | 88.3 | |
| GPT-4o + CoTBackbone=GPT-4o, Prompting Strategy=CoT2025.02 | 88.1 | |
| Qwen3Model Size=8B, Mode=Thinking2026.02 | 87.52 | |
| MirrorModel Size=8B, Mode=Thinking2026.02 | 86.37 | |
| Qwen3Model Size=32B, Mode=No-Thinking2026.02 | 85.73 | |
| Llama-3-70B + QuaSARBackbone=Llama-3-70B, Prompting Strategy=QuaSAR2025.02 | 85.7 | |
| Qwen2-72B + QuaSARBackbone=Qwen2-72B, Prompting Strategy=QuaSAR2025.02 | 83.5 | |
| Qwen3Model Size=14B, Mode=No-Thinking2026.02 | 82.04 | |
| Llama-3-70B + CoTBackbone=Llama-3-70B, Prompting Strategy=CoT2025.02 | 82 | |
| Qwen2-72B + CoMATBackbone=Qwen2-72B, Prompting Strategy=CoMAT2025.02 | 81.7 | |
| GPT-4oBackbone=GPT-4o, Prompting Strategy=Zero-shot2025.02 | 79.7 | |
| Qwen3Model Size=8B, Mode=No-Thinking2026.02 | 79.52 | |
| Qwen2-72B + CoTBackbone=Qwen2-72B, Prompting Strategy=CoT2025.02 | 79.5 | |
| MirrorModel Size=8B, Mode=No-Thinking2026.02 | 79.13 | |
| Raon-Speech2026.04 | 78.87 | |
| GPT-4o + FCoTBackbone=GPT-4o, Prompting Strategy=FCoT2025.02 | 76.8 | |
| Fun-Audio Chat2026.04 | 74.7 | |
| MiniCPM-o 4.52026.04 | 72.53 | |
| HyperCLOVA X 8B Omni2026.04 | 71.83 | |
| Llama-3-70BBackbone=Llama-3-70B, Prompting Strategy=Zero-shot2025.02 | 70.8 | |
| GaC + SAFEModels used in Ensemble=Qwen2 + Llama3.12025.10 | 69.99 | |
| UniTE + SAFEModels used in Ensemble=Qwen2 + Llama3.12025.10 | 69.71 | |
| GaCModels used in Ensemble=Qwen2 + Llama3.12025.10 | 69.5 | |
| Qwen2-7B2025.10 | 69.25 | |
| UniTEModels used in Ensemble=Qwen2 + Llama3.12025.10 | 68.9 | |
| Llama3.1-8B2025.10 | 68.51 | |
| Qwen2.5-Omni2026.04 | 68.03 | |
| Qwen2-72BBackbone=Qwen2-72B, Prompting Strategy=Zero-shot2025.02 | 66.5 | |
| RCOBase Model=LLaMA-3-8B-Instruct2025.06 | 61.3 | |
| LLaMA-3-70B-Instruct2025.06 | 61.24 | |
| RCOBase Model=LLaMA-2-13B-Chat2025.06 | 57.98 | |
| RCOBase Model=UltraCM-13B2025.06 | 57.89 | |
| RCOBase Model=LLaMA-2-7B-Chat2025.06 | 57.67 | |
| RCOBase Model=Auto-J-13B2025.06 | 56.47 | |
| DPCOBase Model=UltraCM-13B2025.06 | 55.94 | |
| DPCOBase Model=Auto-J-13B2025.06 | 53.88 | |
| DPCOBase Model=LLaMA-2-7B-Chat2025.06 | 53.81 | |
| LLAMA-2-70B-Chat2025.06 | 53.55 | |
| DPCOBase Model=LLaMA-3-8B-Instruct2025.06 | 53.52 | |
| LLaMA-3-8B-Instruct2025.06 | 53.24 | |
| DPCOBase Model=LLaMA-2-13B-Chat2025.06 | 53.04 | |
| Interactive 2 mini2026.04 | 51.73 | |
| UltraCM-13B2025.06 | 50.9 | |
| LLaMA-2-7B-Chat2025.06 | 50.82 | |
| LLaMA-2-13B-Chat2025.06 | 50.23 | |
| Self-refinement2025.06 | 49.52 | |
| Initial Answer2025.06 | 48.79 | |
| Aligner2025.06 | 48.59 | |
| Auto-J-13B2025.06 | 46.63 | |
| Kimi Audio2026.04 | 44.27 | |
| Fun-Audio Omni2026.04 | 36.03 | |
| Step-Audio Flamingo 32026.04 | 0.9 |