Mathematical Reasoning on GSM-sym
90.22Exact MatchR1-Oracle
Evaluation Results
| Method | Links | ||||
|---|---|---|---|---|---|
| R1-OracleBackbone=Qwen2.5-3B-Instruct, Supervision Paradigm=100% GT2026.03 | 90.22 | 79.12 | 0 | — | |
| SA-GRPOBase Model=Qwen3-4B, alpha=0.22025.09 | 87.64 | — | — | 323 | |
| GRPO-O1Base Model=Qwen3-4B, alpha=0.052025.09 | 86.76 | — | — | 350 | |
| MemRewardBackbone=Qwen2.5-3B-Instruct, Supervision Paradigm=20% GT + GNN2026.03 | 86.44 | 77.02 | -2.1 | — | |
| GRPOBase Model=Qwen3-4B2025.09 | 86.3 | — | — | 414 | |
| PPOBase Model=Qwen3-4B2025.09 | 85.42 | — | — | 485 | |
| GRPO-ERBase Model=Qwen3-4B, alpha=0.052025.09 | 85.24 | — | — | 297 | |
| R1-pBackbone=Qwen2.5-3B-Instruct, Supervision Paradigm=20% GT2026.03 | 84.67 | 75.67 | -3.45 | — | |
| SA-PPOBase Model=Qwen3-4B2025.09 | 83.52 | — | — | 345 | |
| SA-GRPOBackbone=Gemma3-4B2025.09 | 79.04 | — | — | 654 | |
| GRPOBackbone=Gemma3-4B2025.09 | 78.68 | — | — | 644 | |
| MemRewardBackbone=Qwen2.5-1.5B-Instruct, Supervision Paradigm=20% GT + GNN2026.03 | 77.78 | 68.1 | -2.37 | — | |
| R1-OracleBackbone=Qwen2.5-1.5B-Instruct, Supervision Paradigm=100% GT2026.03 | 75.33 | 70.47 | 0 | — | |
| GRPO-O1Backbone=Gemma3-4B2025.09 | 75.02 | — | — | 464 | |
| GRPOBase Model=Phi-3.5-mini2025.09 | 73.76 | — | — | 337 | |
| SA-PPOBase Model=Phi-3.5-mini2025.09 | 73.52 | — | — | 344 | |
| GRPO-ERBackbone=Gemma3-4B2025.09 | 73.36 | — | — | 623 | |
| SA-GRPOBase Model=Phi-3.5-mini, alpha=0.22025.09 | 72.72 | — | — | 289 | |
| SA-PPOBase Model=Qwen3-1.7B2025.09 | 68.02 | — | — | 369 | |
| GRPOBase Model=Qwen3-1.7B2025.09 | 67.66 | — | — | 413 | |
| SA-GRPOBase Model=Qwen3-1.7B, alpha=0.22025.09 | 67.66 | — | — | 346 | |
| Qwen3-4BBase Model=Qwen3-4B2025.09 | 66.22 | — | — | 435 | |
| GRPO-O1Base Model=Qwen3-1.7B, alpha=0.052025.09 | 65.78 | — | — | 363 | |
| PPOBase Model=Phi-3.5-mini2025.09 | 64.88 | — | — | 322 | |
| GRPO-ERBase Model=Qwen3-1.7B, alpha=0.052025.09 | 64.74 | — | — | 320 | |
| GRPO-ERBase Model=Phi-3.5-mini, alpha=0.052025.09 | 63.54 | — | — | 240 | |
| R1-pBackbone=Qwen2.5-1.5B-Instruct, Supervision Paradigm=20% GT2026.03 | 62.89 | 62.72 | -7.75 | — | |
| PPOBase Model=Qwen3-1.7B2025.09 | 62.88 | — | — | 460 | |
| GRPO-O1Base Model=Phi-3.5-mini, alpha=0.052025.09 | 62.24 | — | — | 279 | |
| Gemma3-4BBackbone=Gemma3-4B, Strategy=Base2025.09 | 57.4 | — | — | 373 | |
| Phi-3.5-miniBase Model=Phi-3.5-mini2025.09 | 56.86 | — | — | 398 | |
| Qwen3-1.7BBase Model=Qwen3-1.7B2025.09 | 46.76 | — | — | 365 | |
| GRPOBase Model=Gemma3-1B2025.09 | 34.7 | — | — | 1,552 | |
| SA-GRPOBase Model=Gemma3-1B, alpha=0.22025.09 | 34.52 | — | — | 772 | |
| GRPO-ERBase Model=Gemma3-1B, alpha=0.052025.09 | 32.36 | — | — | 1,063 | |
| GRPO-O1Base Model=Gemma3-1B, alpha=0.052025.09 | 32.06 | — | — | 1,157 | |
| SA-PPOBase Model=Gemma3-1B2025.09 | 31 | — | — | 936 | |
| PPOBase Model=Gemma3-1B2025.09 | 30.28 | — | — | 1,187 | |
| GRPOBackbone=Qwen2-2B-VL2025.09 | 27.72 | — | — | 209 | |
| SA-GRPOBackbone=Qwen2-2B-VL2025.09 | 27.66 | — | — | 174 | |
| GRPO-ERBackbone=Qwen2-2B-VL2025.09 | 26.52 | — | — | 179 | |
| GRPO-O1Backbone=Qwen2-2B-VL2025.09 | 25.44 | — | — | 164 | |
| Gemma3-1BBase Model=Gemma3-1B2025.09 | 19.58 | — | — | 427 | |
| Qwen2-2B-VLBackbone=Qwen2-2B-VL, Strategy=Base2025.09 | 5.98 | — | — | 1,873 |