Mathematical Reasoning on GSM8K, MATH500, AIME25, AIME26, AMC, Olympiad
92.4GSM8K ScoreLUFFY (Gemini)
Evaluation results
| Method | Links | |||||||
|---|---|---|---|---|---|---|---|---|
| LUFFY (Gemini)Backbone=Qwen3-8B-Base, Expert Trace Source=Gemini-3-Pro2026.05 | 92.4 | 74.6 | 15.2 | 14.7 | 65 | 30.7 | 48.8 | |
| SMEPO (Gemini)Backbone=Qwen3-8B-Base, Expert Trace Source=Gemini-3-Pro2026.05 | 92.3 | 78.6 | 16.1 | 14.4 | 67.5 | 32.8 | 50.3 | |
| SMEPO (DS)Backbone=Qwen3-8B-Base, Expert Trace Source=DeepSeek-R12026.05 | 92.2 | 76.6 | 17.6 | 13.4 | 70 | 32 | 50.3 | |
| SFT (DS) + RLBackbone=Qwen3-8B-Base, Expert Trace Source=DeepSeek-R1, Strategy=SFT + RL2026.05 | 91.7 | 76.8 | 15 | 10.4 | 62.5 | 29.7 | 47.7 | |
| SFT (Gemini)Backbone=Qwen3-8B-Base, Expert Trace Source=Gemini-3-Pro, Strategy=SFT2026.05 | 91.7 | 71.2 | 10.2 | 9 | 52.5 | 25.2 | 43.3 | |
| SFT (Gemini) + RLBackbone=Qwen3-8B-Base, Expert Trace Source=Gemini-3-Pro, Strategy=SFT + RL2026.05 | 91.6 | 73 | 13.2 | 10.6 | 60 | 28 | 46.1 | |
| GRPOBackbone=Qwen3-8B-Base, Strategy=Vanilla GRPO2026.05 | 91.3 | 75.2 | 12.5 | 12.3 | 65 | 29.5 | 47.6 | |
| LUFFY (DS)Backbone=Qwen3-8B-Base, Expert Trace Source=DeepSeek-R12026.05 | 90.4 | 74.6 | 14.4 | 12.6 | 60 | 29.2 | 46.9 | |
| SMEPO (DS)Backbone=Qwen2.5-7B, Expert Trace Source=DeepSeek-R12026.05 | 89 | 72.4 | 10.8 | 9.7 | 60 | 25.1 | 44.5 | |
| SFT (DS)Backbone=Qwen3-8B-Base, Expert Trace Source=DeepSeek-R1, Strategy=SFT2026.05 | 88 | 71.2 | 8 | 5.6 | 50 | 25.4 | 41.4 | |
| GRPOBackbone=Qwen2.5-7B, Strategy=Vanilla GRPO2026.05 | 87.6 | 70.2 | 9.4 | 7.7 | 55 | 24.3 | 42.4 | |
| SMEPO (DS)Backbone=DeepSeek-R1-Distill-Qwen-7B, Expert Trace Source=DeepSeek-R12026.05 | 87.5 | 85.4 | 34.7 | 39.7 | 90 | 38.3 | 62.6 | |
| Expert (DS)Backbone=DeepSeek-R1-Distill-Qwen-7B, Expert Trace Source=DeepSeek-R12026.05 | 87.3 | 81 | 34.1 | 38.4 | 85 | 36.8 | 60.4 | |
| SFT (DS)Backbone=Qwen2.5-7B, Expert Trace Source=DeepSeek-R12026.05 | 87 | 66.8 | 7.3 | 5 | 42.5 | 22.1 | 38.5 | |
| GRPOBackbone=DeepSeek-R1-Distill-Qwen-7B, Strategy=Vanilla GRPO2026.05 | 86.4 | 84.2 | 34 | 38.8 | 90 | 37.2 | 61.8 | |
| Expert (DS)Backbone=Qwen2.5-7B, Expert Trace Source=DeepSeek-R12026.05 | 85.3 | 63.8 | 5.6 | 6.4 | 47.5 | 22.8 | 38.6 | |
| R1-Distill-Qwen-7BBackbone=DeepSeek-R1-Distill-Qwen-7B, Stage=Base Model2026.05 | 83.5 | 81 | 32.6 | 33.3 | 82.5 | 35.3 | 58 | |
| GHPO (DS)Backbone=Qwen3-8B-Base, Expert Trace Source=DeepSeek-R12026.05 | 79.5 | 73.6 | 13.5 | 10 | 60 | 29.2 | 44.3 | |
| GHPO (Gemini)Backbone=Qwen3-8B-Base, Expert Trace Source=Gemini-3-Pro2026.05 | 79.4 | 73.2 | 13.9 | 9.5 | 65 | 29.5 | 45.1 | |
| Expert (Gemini)Backbone=Qwen3-8B-Base, Expert Trace Source=Gemini-3-Pro, Strategy=Naive expert trace2026.05 | 78 | 68.8 | 11.6 | 9.6 | 50 | 26.6 | 40.8 | |
| Expert (DS)Backbone=Qwen3-8B-Base, Expert Trace Source=DeepSeek-R1, Strategy=Naive expert trace2026.05 | 76.4 | 69.2 | 11.4 | 8.6 | 52.5 | 27.7 | 41 | |
| Qwen3-8B-BaseBackbone=Qwen3-8B-Base, Stage=Base Model2026.05 | 68 | 67 | 9.9 | 7.6 | 50 | 25.8 | 38.1 | |
| Qwen2.5-7BBackbone=Qwen2.5-7B, Stage=Base Model2026.05 | 46.2 | 55.8 | 4.2 | 5.4 | 35 | 20 | 27.8 |