Science Reasoning on ARC
89.6AccuracyMENTOR
Evaluation results
| Method | Links | |
|---|---|---|
| MENTORBackbone=Qwen2.5-7B-Base2025.10 | 89.6 | |
| QuestABackbone=Qwen2.5-7B-Base2025.10 | 86.7 | |
| On-policy RLBackbone=Qwen2.5-7B-Base2025.10 | 86 | |
| LUFFYBackbone=Qwen2.5-7B-Base2025.10 | 86 | |
| GRPO (RLVRR)#Data=10K2026.01 | 84.9 | |
| GRPO (RM)#Data=10K2026.01 | 84.2 | |
| DPO#Data=10K2026.01 | 83.8 | |
| SFT#Data=10K2026.01 | 83.7 | |
| Instruct#Data=n/a2026.01 | 83.4 | |
| GRPO (GRM)#Data=10K2026.01 | 83.2 | |
| GRPO (RLPR)#Data=10K2026.01 | 82.8 | |
| GRPO (BLEU)#Data=10K2026.01 | 82 | |
| SFT#Data=100K2026.01 | 81.6 | |
| MENTORBackbone=Qwen2.5-3B-Base2025.10 | 80.8 | |
| GRPO (Random)#Data=10K2026.01 | 79.7 | |
| MENTORBackbone=LLaMa3.1-8B-Base2025.10 | 77.3 | |
| LUFFYBackbone=LLaMa3.1-8B-Base2025.10 | 74.9 | |
| QuestABackbone=LLaMa3.1-8B-Base2025.10 | 72.5 | |
| LUFFYBackbone=Qwen2.5-3B-Base2025.10 | 72.5 | |
| On-policy RLBackbone=Qwen2.5-3B-Base2025.10 | 72.1 | |
| On-policy RLBackbone=LLaMa3.1-8B-Base2025.10 | 70.7 | |
| BaseBackbone=Qwen2.5-7B-Base2025.10 | 70.4 | |
| QuestABackbone=Qwen2.5-3B-Base2025.10 | 70.3 | |
| BaseBackbone=Qwen2.5-3B-Base2025.10 | 23.6 | |
| BaseBackbone=LLaMa3.1-8B-Base2025.10 | 0 |