Question Answering on MMLU-Pro (Natural Setting Test)
87.8AccuracySA (Base)
Evaluation results
| Method | Links | |
|---|---|---|
| SA (Base)Backbone=Gemini 3 Pro2026.01 | 87.8 | |
| MA-Outcome (AG)Backbone=Gemini 3 Pro2026.01 | 87.8 | |
| MA-Reasoning (ECL E)Backbone=Qwen 3-30B2026.01 | 86.7 | |
| MA-Outcome (ECL I)Backbone=Deepseek V3.22026.01 | 86.7 | |
| MA-Reasoning (AG)Backbone=GPT-5-mini2026.01 | 86.7 | |
| MA-Reasoning (ECL E)Backbone=GPT-5-mini2026.01 | 86.7 | |
| MA-Outcome (ECL E)Backbone=GPT-5.22026.01 | 86.7 | |
| MA-Reasoning (ECL E)Backbone=GPT-5.22026.01 | 86.7 | |
| MA-Outcome (ECL I)Backbone=Gemini 3 Flash2026.01 | 86.7 | |
| MA-Reasoning (ECL E)Backbone=Gemini 3 Pro2026.01 | 86.7 | |
| MA-Reasoning (ECL I)Backbone=Qwen 3-4B2026.01 | 85.6 | |
| MA-Outcome (ECL E)Backbone=GPT-5-mini2026.01 | 85.6 | |
| MA-Reasoning (ECL I)Backbone=GPT-5-mini2026.01 | 85.6 | |
| MA-Outcome (AG)Backbone=GPT-5.22026.01 | 85.6 | |
| MA-Reasoning (AG)Backbone=GPT-5.22026.01 | 85.6 | |
| SA (Base)Backbone=Gemini 3 Flash2026.01 | 85.6 | |
| MA-Outcome (ECL E)Backbone=Gemini 3 Flash2026.01 | 85.6 | |
| MA-Reasoning (ECL E)Backbone=Gemini 3 Flash2026.01 | 85.6 | |
| MA-Outcome (ECL I)Backbone=Gemini 3 Pro2026.01 | 85.6 | |
| MA-Outcome (ECL E)Backbone=Gemini 3 Pro2026.01 | 85.6 | |
| MA-Reasoning (AG)Backbone=Gemini 3 Pro2026.01 | 85.6 | |
| MA-Reasoning (ECL I)Backbone=Gemini 3 Pro2026.01 | 85.6 | |
| MA-Reasoning (ECL E)Backbone=Qwen 3-4B2026.01 | 84.4 | |
| MA-Outcome (ECL E)Backbone=Deepseek V3.22026.01 | 84.4 | |
| MA-Outcome (ECL I)Backbone=GPT-5-mini2026.01 | 84.4 | |
| SA (Base)Backbone=GPT-5.22026.01 | 84.4 | |
| MA-Reasoning (ECL I)Backbone=GPT-5.22026.01 | 84.4 | |
| MA-Reasoning (ECL I)Backbone=Gemini 3 Flash2026.01 | 84.4 | |
| MA-Reasoning (AG)Backbone=Deepseek V3.22026.01 | 83.3 | |
| MA-Reasoning (ECL I)Backbone=Deepseek V3.22026.01 | 83.3 | |
| MA-Reasoning (ECL E)Backbone=Deepseek V3.22026.01 | 83.3 | |
| MA-Outcome (ECL I)Backbone=GPT-5.22026.01 | 83.3 | |
| MA-Reasoning (AG)Backbone=Gemini 3 Flash2026.01 | 83.3 | |
| MA-Reasoning (AG)Backbone=Qwen 3-30B2026.01 | 82.2 | |
| MA-Reasoning (ECL I)Backbone=Qwen 3-30B2026.01 | 82.2 | |
| MA-Outcome (AG)Backbone=GPT-5-mini2026.01 | 82.2 | |
| MA-Outcome (AG)Backbone=Gemini 3 Flash2026.01 | 82.2 | |
| MA-Outcome (AG)Backbone=Deepseek V3.22026.01 | 81.1 | |
| SA (Base)Backbone=Deepseek V3.22026.01 | 80 | |
| SA (Base)Backbone=GPT-5-mini2026.01 | 80 | |
| MA-Reasoning (AG)Backbone=Qwen 3-4B2026.01 | 78.9 | |
| MA-Outcome (ECL E)Backbone=Qwen 3-8B2026.01 | 77.8 | |
| MA-Outcome (ECL I)Backbone=Qwen 3-30B2026.01 | 77.8 | |
| MA-Outcome (ECL E)Backbone=Qwen 3-30B2026.01 | 77.8 | |
| MA-Reasoning (ECL E)Backbone=Qwen 3-8B2026.01 | 76.7 | |
| SA (Base)Backbone=Qwen 3-30B2026.01 | 75.6 | |
| MA-Outcome (AG)Backbone=Qwen 3-30B2026.01 | 73.3 | |
| MA-Outcome (ECL I)Backbone=Qwen 3-4B2026.01 | 72.2 | |
| MA-Reasoning (ECL I)Backbone=Qwen 3-8B2026.01 | 72.2 | |
| MA-Outcome (ECL E)Backbone=Qwen 3-4B2026.01 | 71.1 | |
| SA (RL)Backbone=Qwen 3-8B2026.01 | 71.1 | |
| MA-Reasoning (AG)Backbone=Qwen 3-8B2026.01 | 71.1 | |
| SA (RL)Backbone=Qwen 3-4B2026.01 | 70 | |
| MA-Outcome (AG)Backbone=Qwen 3-4B2026.01 | 67.8 | |
| MA-Outcome (AG)Backbone=Qwen 3-8B2026.01 | 66.7 | |
| MA-Outcome (ECL I)Backbone=Qwen 3-8B2026.01 | 65.6 |