General Reasoning on MMLU-Pro (pass@1 accuracy)
73.44pass@1 AccuracyFP16
Evaluation results
| Method | Links | |
|---|---|---|
| FP16Model=Qwen3-8B, Quantization Bits=FP162026.05 | 73.44 | |
| CLPOOptimization Policy=Restructuring-Aware RL, Base Model=Qwen3-8B2025.09 | 73.12 | |
| Critique-GRPO (CoT)Optimization Policy=Critique-Driven RL, Base Model=Qwen3-8B2025.09 | 70.98 | |
| Critique-GRPO (CoT Critique)Backbone=Qwen3-8B2025.06 | 70.47 | |
| LUFFYOptimization Policy=Off-Policy Imitation, Base Model=Qwen3-8B2025.09 | 70.34 | |
| FP16Model=Qwen3-4B, Quantization Bits=FP162026.05 | 70.23 | |
| Critique-GRPO (Simple)Optimization Policy=Critique-Driven RL, Base Model=Qwen3-8B2025.09 | 70.13 | |
| DAPOOptimization Policy=Dynamic Sampling, Base Model=Qwen3-8B2025.09 | 70.01 | |
| GRPOOptimization Policy=Group-Based RL, Base Model=Qwen3-8B2025.09 | 69.86 | |
| ParoQModel=Qwen3-8B, Quantization Bits=3-bit2026.05 | 68.93 | |
| R-QATModel=Qwen3-8B, Quantization Bits=3-bit2026.05 | 68.33 | |
| Qwen3-8BBackbone=Qwen3-8B2025.06 | 68.25 | |
| Refinement-FTOptimization Policy=Guided Refinement, Base Model=Qwen3-8B2025.09 | 67.84 | |
| LAQuantModel=Qwen3-8B, Quantization Bits=3-bit2026.05 | 66.86 | |
| CITL-FTOptimization Policy=Mixed-Data SFT, Base Model=Qwen3-8B2025.09 | 66.13 | |
| RAFTOptimization Policy=Ranking-Based Imitation, Base Model=Qwen3-8B2025.09 | 65.93 | |
| Rule-basedBackbone=Qwen3-8B-Base, Training Dataset=mathematical dataset2026.03 | 65.8 | |
| GPTQModel=Qwen3-8B, Quantization Bits=3-bit2026.05 | 65.71 | |
| Rule+CERBackbone=Qwen3-8B-Base, Training Dataset=mathematical dataset2026.03 | 65.6 | |
| CERBackbone=Qwen3-8B-Base, Training Dataset=mathematical dataset2026.03 | 64.8 | |
| General-verifierBackbone=Qwen3-8B-Base, Training Dataset=mathematical dataset2026.03 | 64.3 | |
| ParoQModel=Qwen3-4B, Quantization Bits=3-bit2026.05 | 63.41 | |
| ParoQ++Model=Qwen3-8B, Quantization Bits=3-bit2026.05 | 63.32 | |
| R-QATModel=Qwen3-4B, Quantization Bits=3-bit2026.05 | 63.24 | |
| Critique-FTOptimization Policy=Learning to Critique, Base Model=Qwen3-8B2025.09 | 63.16 | |
| AEROBackbone Model=Qwen3-8B-Base, Evolutionary Round=R52026.02 | 62.8 | |
| LAQuantModel=Qwen3-4B, Quantization Bits=3-bit2026.05 | 62.63 | |
| Absolute ZeroBackbone Model=Qwen3-8B-Base2026.02 | 62.5 | |
| Rule-basedBackbone=Qwen3-4B-Base, Training Dataset=mathematical dataset2026.03 | 62.5 | |
| ParoQ++Model=Qwen3-4B, Quantization Bits=3-bit2026.05 | 62.41 | |
| VeriFreeBackbone=Qwen3-8B-Base, Training Dataset=mathematical dataset2026.03 | 62.3 | |
| R-ZeroBackbone Model=Qwen3-8B-Base2026.02 | 61.6 | |
| AEROBackbone Model=Qwen3-8B-Base, Evolutionary Round=R42026.02 | 61.5 | |
| Exact-matchBackbone=Qwen3-8B-Base, Training Dataset=mathematical dataset2026.03 | 61.4 | |
| Rule+CERBackbone=Qwen3-4B-Base, Training Dataset=mathematical dataset2026.03 | 61.2 | |
| CERBackbone=Qwen3-4B-Base, Training Dataset=mathematical dataset2026.03 | 60.8 | |
| AEROBackbone Model=Qwen3-8B-Base, Evolutionary Round=R22026.02 | 60.3 | |
| AEROBackbone Model=Qwen3-8B-Base, Evolutionary Round=R32026.02 | 60.3 | |
| VeriFreeBackbone=Qwen3-4B-Base, Training Dataset=mathematical dataset2026.03 | 59.9 | |
| AEROBackbone Model=Qwen3-8B-Base, Evolutionary Round=R12026.02 | 59.3 | |
| FP16Model=R1-Distill-Llama-8B, Quantization Bits=FP162026.05 | 58.84 | |
| General-verifierBackbone=Qwen3-4B-Base, Training Dataset=mathematical dataset2026.03 | 58.5 | |
| Qwen3-8B-BaseBackbone Model=Qwen3-8B-Base2026.02 | 58 | |
| FP16Model=Qwen3-1.7B, Quantization Bits=FP162026.05 | 57.31 | |
| AEROBackbone Model=Qwen3-4B-Base, Evolutionary Round=R52026.02 | 56.9 | |
| Absolute ZeroBackbone Model=Qwen3-4B-Base2026.02 | 56.2 | |
| AEROBackbone Model=Qwen3-4B-Base, Evolutionary Round=R42026.02 | 55.6 | |
| Critique-GRPO (CoT Critique)Backbone=Qwen2.5-7B-Base2025.06 | 55.28 | |
| Continual LUFFYBackbone=Qwen2.5-Math-7B, RLVR Setup=Continual RLVR2025.10 | 54.7 | |
| ExGRPO (Continual)Backbone=Qwen2.5-Math-7B, RLVR Setup=Continual RLVR2025.10 | 54.5 | |
| R-ZeroBackbone Model=Qwen3-4B-Base2026.02 | 54.2 | |
| AEROBackbone Model=Qwen3-4B-Base, Evolutionary Round=R32026.02 | 53.9 | |
| GPTQModel=Qwen3-4B, Quantization Bits=3-bit2026.05 | 53.32 | |
| AEROBackbone Model=Qwen3-4B-Base, Evolutionary Round=R22026.02 | 53.3 | |
| On-Policy (Continual)Backbone=Qwen2.5-Math-7B, RLVR Setup=Continual RLVR2025.10 | 53.3 | |
| LUFFYBackbone=Qwen2.5-Math-7B, RLVR Setup=Continual RLVR2025.10 | 53 | |
| ExGRPOBackbone=Qwen2.5-Math-7B, RLVR Setup=Zero RLVR2025.10 | 52.9 | |
| AEROBackbone Model=Qwen3-4B-Base, Evolutionary Round=R12026.02 | 52.4 | |
| DPSBackbone=R1-Distill-7B, Finetuning=true, Rollouts=287k, Runtime=39h, Max response length=8k tokens, Training dataset=MATH2026.03 | 52.37 | |
| BaseBackbone=Qwen3-8B-Base, Training Dataset=mathematical dataset2026.03 | 51.9 | |
| LAQuantModel=R1-Distill-Llama-8B, Quantization Bits=3-bit2026.05 | 51.7 | |
| Qwen3-4B-BaseBackbone Model=Qwen3-4B-Base2026.02 | 51.6 | |
| HRBackbone=R1-Distill-7B, Finetuning=true, Rollouts=287k, Runtime=36h, Max response length=8k tokens, Training dataset=MATH2026.03 | 51.56 | |
| ParoQ++Model=R1-Distill-Llama-8B, Quantization Bits=3-bit2026.05 | 51.44 | |
| DS (Oracle)Backbone=R1-Distill-7B, Finetuning=true, Rollouts=1147k, Runtime=73h, Max response length=8k tokens, Training dataset=MATH2026.03 | 51.43 | |
| R-QATModel=R1-Distill-Llama-8B, Quantization Bits=3-bit2026.05 | 51.05 | |
| USBackbone=R1-Distill-7B, Finetuning=true, Rollouts=287k, Runtime=30h, Max response length=8k tokens, Training dataset=MATH2026.03 | 50.59 | |
| GPG-ZeroBackbone=Qwen2.5-Math-7B, RLVR Setup=Previous Zero RLVR2025.10 | 50.5 | |
| R1-Distill-7BBackbone=R1-Distill-7B, Max response length=8k tokens, Training dataset=MATH2026.03 | 50.44 | |
| On-PolicyBackbone=Qwen2.5-Math-7B, RLVR Setup=Zero RLVR2025.10 | 49.2 | |
| LAQuantModel=Qwen3-1.7B, Quantization Bits=3-bit2026.05 | 47.94 | |
| R-QATModel=Qwen3-1.7B, Quantization Bits=3-bit2026.05 | 47.48 | |
| GPTQModel=R1-Distill-Llama-8B, Quantization Bits=3-bit2026.05 | 47.1 | |
| Exact-matchBackbone=Qwen3-4B-Base, Training Dataset=mathematical dataset2026.03 | 46.7 | |
| CoVRLBackbone=Qwen2.5-7B-Base2025.12 | 46.5 | |
| Qwen2.5-7B-BaseBackbone=Qwen2.5-7B-Base2025.06 | 46.24 | |
| ParoQ++Model=Qwen3-1.7B, Quantization Bits=3-bit2026.05 | 45.84 | |
| RLPRBackbone=Qwen2.5-7B-Base, Algorithm=GRPO2025.12 | 44.9 | |
| RAVRBackbone=Qwen2.5-7B-Base, Algorithm=GRPO2025.12 | 44.5 | |
| ParoQModel=R1-Distill-Llama-8B, Quantization Bits=3-bit2026.05 | 44.29 | |
| VeriFreeBackbone=Qwen2.5-7B-Base, Algorithm=GRPO2025.12 | 44.1 | |
| ParoQModel=Qwen3-1.7B, Quantization Bits=3-bit2026.05 | 43.79 | |
| EOPDStudent Model=Qwen3-1.7B-Base2026.03 | 43.2 | |
| JLBBackbone=Qwen2.5-7B-Base, Algorithm=GRPO2025.12 | 42.7 | |
| LaTROBackbone=Qwen2.5-7B-Base, Algorithm=GRPO2025.12 | 42.7 | |
| SFTBackbone=Qwen2.5-Math-7B, RLVR Setup=Off-policy Learning2025.10 | 42.7 | |
| RePO-ZeroBackbone=Qwen2.5-Math-7B, RLVR Setup=Previous Zero RLVR2025.10 | 42.5 | |
| OPDStudent Model=Qwen3-1.7B-Base2026.03 | 42.26 | |
| BaseBackbone=Qwen3-4B-Base, Training Dataset=mathematical dataset2026.03 | 42 | |
| Oat-ZeroBackbone=Qwen2.5-Math-7B, RLVR Setup=Previous Zero RLVR2025.10 | 41.7 | |
| GRPOStudent Model=Qwen3-1.7B-Base2026.03 | 41.46 | |
| SFT+RLBackbone=Qwen2.5-Math-7B, RLVR Setup=Off-policy Learning2025.10 | 37.7 | |
| KDStudent Model=Qwen3-1.7B-Base2026.03 | 37.54 | |
| Base ModelBackbone=Qwen2.5-7B-Base2025.12 | 36.7 | |
| OpenSIRBackbone=Llama-3.1-8B-Instruct2025.11 | 36.48 | |
| GRPOgsm8kBackbone=Llama-3.1-8B-Instruct2025.11 | 34.18 | |
| Qwen-InstructBackbone=Qwen2.5-Math-7B2025.10 | 34.1 | |
| GRPOmathBackbone=Llama-3.1-8B-Instruct2025.11 | 33.27 | |
| Absolute ZeroBackbone=Llama-3.1-8B-Instruct2025.11 | 33.08 | |
| R-ZeroBackbone=Llama-3.1-8B-Instruct2025.11 | 32.92 |