Reasoning on MMLU-Pro (Pass@1 and Pass@4)
71Pass@1Rule+CER
Evaluation results
| Method | Links | ||
|---|---|---|---|
| Rule+CERBackbone=Qwen3-8B-Base, Reward Method=Rule+CER, Training Data Domain=General-domain2026.03 | 71 | — | |
| CERBackbone=Qwen3-8B-Base, Reward Method=CER, Training Data Domain=General-domain2026.03 | 69.7 | — | |
| General-verifierBackbone=Qwen3-8B-Base, Reward Method=General-verifier, Training Data Domain=General-domain2026.03 | 67.3 | — | |
| Rule-basedBackbone=Qwen3-8B-Base, Reward Method=Rule-based, Training Data Domain=General-domain2026.03 | 66.6 | — | |
| Exact-MatchBackbone=Qwen3-8B-Base, Reward Method=Exact-Match, Training Data Domain=General-domain2026.03 | 66.2 | — | |
| VeriFreeBackbone=Qwen3-8B-Base, Reward Method=VeriFree, Training Data Domain=General-domain2026.03 | 65.4 | — | |
| Rule+CERBackbone=Qwen3-4B-Base, Reward Method=Rule+CER, Training Data Domain=General-domain2026.03 | 64.1 | — | |
| CERBackbone=Qwen3-4B-Base, Reward Method=CER, Training Data Domain=General-domain2026.03 | 63.8 | — | |
| General-verifierBackbone=Qwen3-4B-Base, Reward Method=General-verifier, Training Data Domain=General-domain2026.03 | 63.7 | — | |
| Exact-MatchBackbone=Qwen3-4B-Base, Reward Method=Exact-Match, Training Data Domain=General-domain2026.03 | 62.9 | — | |
| Rule-basedBackbone=Qwen3-4B-Base, Reward Method=Rule-based, Training Data Domain=General-domain2026.03 | 62.3 | — | |
| PSFTwarm-upBackbone=Qwen2.5-7B-Instruct2025.08 | 59.28 | — | |
| PSFTBackbone=Qwen2.5-7B-Instruct2025.08 | 59.18 | — | |
| SFTBackbone=Qwen2.5-7B-Instruct2025.08 | 58.98 | — | |
| VeriFreeBackbone=Qwen3-4B-Base, Reward Method=VeriFree, Training Data Domain=General-domain2026.03 | 58.7 | — | |
| SFT-KLBackbone=Qwen2.5-7B-Instruct2025.08 | 58.35 | — | |
| PSFTBackbone=Llama3.1-8B-Instruct2025.08 | 56.58 | — | |
| PSFTwarm-upBackbone=Llama3.1-8B-Instruct2025.08 | 56.03 | — | |
| EVOL-RLBase Model=Qwen3-8B-Base, Training=Label-free training on MATH-TRAIN2025.09 | 55.3 | 78.5 | |
| BaseBackbone=Qwen2.5-7B-Instruct2025.08 | 54.99 | — | |
| TTRLBase Model=Qwen3-8B-Base, Training=Label-free training on MATH-TRAIN2025.09 | 53.4 | 73.9 | |
| BaseBackbone=Qwen3-8B-Base, Reward Method=None, Training Data Domain=General-domain2026.03 | 51.9 | — | |
| SFTBackbone=Llama3.1-8B-Instruct2025.08 | 50.29 | — | |
| Qwen3-8B-BaseTraining=None (Base Model)2025.09 | 47.3 | 74.5 | |
| SFT-KLBackbone=Llama3.1-8B-Instruct2025.08 | 47.02 | — | |
| BaseBackbone=Llama3.1-8B-Instruct2025.08 | 43.7 | — | |
| BaseBackbone=Qwen3-4B-Base, Reward Method=None, Training Data Domain=General-domain2026.03 | 42 | — |