Mathematical Reasoning on MATH500 (Performance (%))
98.6Performance (%)Gemini-2.5-pro
Evaluation Results
| Method | Links | |
|---|---|---|
| Gemini-2.5-proCategory=Proprietary LLMs2026.06 | 98.6 | |
| MiniMax-M2.5Category=Proprietary LLMs2026.06 | 98.4 | |
| Qwen3-8B-OPERACategory=Our Methods, Technique=OPERA2026.06 | 96.2 | |
| Qwen3-8BCategory=Our Methods2026.06 | 95.6 | |
| Qwen3-8B-SFTCategory=Our Methods2026.06 | 94.8 | |
| Ouro 2.6B R4Architecture=LoopLM, # Total Params=2.6B, # Trained Tokens=7.7T2025.10 | 90.85 | |
| VRPOModel Scale=Qwen3-8B, Optimization Method=VRPO2025.08 | 90.2 | |
| VRPOBase Model=Qwen3-8B2025.08 | 90.2 | |
| λ-GRPOModel Scale=Qwen3-8B, Optimization Method=λ-GRPO2025.08 | 90 | |
| λ-GRPOBase Model=Qwen3-8B2025.08 | 90 | |
| GRPOModel Scale=Qwen3-8B, Optimization Method=GRPO2025.08 | 89.2 | |
| Reinforce++Model Scale=Qwen3-8B, Optimization Method=Reinforce++2025.08 | 89 | |
| Dr.GRPOModel Scale=Qwen3-8B, Optimization Method=Dr.GRPO2025.08 | 87.8 | |
| BaseModel Scale=Qwen3-8B, Optimization Method=Base2025.08 | 87.4 | |
| BaseBase Model=Qwen3-8B2025.08 | 87.4 | |
| PPOModel Scale=Qwen3-8B, Optimization Method=PPO2025.08 | 86.4 | |
| KTAEModel Scale=Qwen3-8B, Optimization Method=KTAE2025.08 | 86 | |
| KTAEBase Model=Qwen3-8B2025.08 | 86 | |
| VRPOModel Scale=Qwen3-1.7B, Optimization Method=VRPO2025.08 | 85.4 | |
| BaseModel Scale=Qwen3-1.7B, Optimization Method=Base2025.08 | 84.8 | |
| Reinforce++Model Scale=Qwen3-1.7B, Optimization Method=Reinforce++2025.08 | 84.4 | |
| GRPOModel Scale=Qwen3-1.7B, Optimization Method=GRPO2025.08 | 84.2 | |
| PPOModel Scale=Qwen3-1.7B, Optimization Method=PPO2025.08 | 83.8 | |
| Dr.GRPOModel Scale=Qwen3-1.7B, Optimization Method=Dr.GRPO2025.08 | 83.4 | |
| Gemma3Architecture=Dense, # Total Params=12.0B, # Trained Tokens=12T2025.10 | 83.2 | |
| λ-GRPOModel Scale=Qwen3-1.7B, Optimization Method=λ-GRPO2025.08 | 82.8 | |
| DeepWriter-8BCategory=Open-source LLMs2026.06 | 80.4 | |
| Entropy-Cut MHModel=Qwen3-8B-Base2026.05 | 80.2 | |
| TMCModel=Qwen3-8B-Base2026.05 | 79.7 | |
| Uniform-Cut MHModel=Qwen3-8B-Base2026.05 | 79.5 | |
| SMCModel=Qwen3-8B-Base2026.05 | 79.3 | |
| Entropy-Cut MHModel=Qwen2.5-Math-7B2026.05 | 79 | |
| SMCModel=Qwen2.5-Math-7B2026.05 | 78.9 | |
| TMCModel=Qwen2.5-Math-7B2026.05 | 78.2 | |
| GPT-4o-0513Category=Proprietary LLMs2026.06 | 78 | |
| Low-TemperatureModel=Qwen3-8B-Base2026.05 | 77.3 | |
| λ-GRPOBase Model=Qwen2.5-7B-Cold Start2025.08 | 74 | |
| CAREBackbone=Phi4-mini-instruct2026.05 | 73.77 | |
| LongWriter-Zero-32BCategory=Open-source LLMs2026.06 | 73.4 | |
| GTBackbone=Phi4-mini-instruct2026.05 | 73.25 | |
| Uniform-Cut MHModel=Qwen2.5-Math-7B2026.05 | 73.1 | |
| VRPOBase Model=Qwen2.5-7B-Cold Start2025.08 | 72.2 | |
| Entropy-Cut MHModel=Qwen2.5-7B2026.05 | 71.9 | |
| BaseBase Model=Qwen2.5-7B-Cold Start2025.08 | 71.6 | |
| TMCModel=Qwen2.5-7B2026.05 | 70.4 | |
| SMCModel=Qwen2.5-7B2026.05 | 69.8 | |
| OracleBackbone=Phi4-mini-instruct2026.05 | 69.73 | |
| Entropy-Cut MHModel=Phi-4-mini-instruct2026.05 | 68.8 | |
| Uniform-Cut MHModel=Phi-4-mini-instruct2026.05 | 68.7 | |
| KTAEBase Model=Qwen2.5-7B-Cold Start2025.08 | 68.6 | |
| Gemma3Architecture=Dense, # Total Params=4.0B, # Trained Tokens=4T2025.10 | 68.6 | |
| Low-TemperatureModel=Qwen2.5-Math-7B2026.05 | 68.3 | |
| SMCModel=Phi-4-mini-instruct2026.05 | 68.1 | |
| ProbBackbone=Phi4-mini-instruct2026.05 | 67.83 | |
| RandomBackbone=Phi4-mini-instruct2026.05 | 67.7 | |
| Uniform-Cut MHModel=Qwen2.5-7B2026.05 | 67.4 | |
| Low-TemperatureModel=Phi-4-mini-instruct2026.05 | 66.4 | |
| TTRLBackbone=Phi4-mini-instruct2026.05 | 66.3 | |
| TMCModel=Phi-4-mini-instruct2026.05 | 66.1 | |
| KTAEModel Scale=Qwen3-1.7B, Optimization Method=KTAE2025.08 | 64.4 | |
| EntropyBackbone=Phi4-mini-instruct2026.05 | 63.95 | |
| Low-TemperatureModel=Qwen2.5-7B2026.05 | 62.3 | |
| Qwen3Architecture=Dense, # Total Params=8.0B, # Trained Tokens=36T2025.10 | 62.3 | |
| Qwen2.5Architecture=Dense, # Total Params=7.0B, # Trained Tokens=18T2025.10 | 61.2 | |
| Qwen3Architecture=Dense, # Total Params=4.0B, # Trained Tokens=36T2025.10 | 59.6 | |
| StandardModel=Qwen3-8B-Base2026.05 | 58.5 | |
| Llama3.1-8B-SFTCategory=Our Methods2026.06 | 58.2 | |
| Llama3.1-8B-OPERACategory=Our Methods, Technique=OPERA2026.06 | 56.8 | |
| StandardModel=Phi-4-mini-instruct2026.05 | 55.5 | |
| Llama3.1Architecture=Dense, # Total Params=8.0B, # Trained Tokens=15T2025.10 | 52.9 | |
| Llama3.1-8BCategory=Our Methods2026.06 | 52.4 | |
| StandardModel=Qwen2.5-Math-7B2026.05 | 50.1 | |
| SMCModel=Phi-3.5-mini-instruct2026.05 | 49.8 | |
| Entropy-Cut MHModel=Phi-3.5-mini-instruct2026.05 | 49.7 | |
| TMCModel=Phi-3.5-mini-instruct2026.05 | 49.3 | |
| Uniform-Cut MHModel=Phi-3.5-mini-instruct2026.05 | 49.1 | |
| Low-TemperatureModel=Phi-3.5-mini-instruct2026.05 | 45.3 | |
| StandardModel=Phi-3.5-mini-instruct2026.05 | 42.9 | |
| Qwen2.5Architecture=Dense, # Total Params=3.0B, # Trained Tokens=18T2025.10 | 42.6 | |
| Llama3.2Architecture=Dense, # Total Params=3.0B, # Trained Tokens=9T2025.10 | 40.8 | |
| VanillaBackbone=Phi4-mini-instruct2026.05 | 39.93 | |
| StandardModel=Qwen2.5-7B2026.05 | 35.9 | |
| LongWriter-8BCategory=Open-source LLMs2026.06 | 20.4 |