Mathematical Reasoning on Math500 (Accuracy (%))
93Accuracy (%)SPIRAL
Evaluation Results
| Method | Links | |
|---|---|---|
| SPIRALBackbone=DeepSeek-Distill-Qwen-7B, Training Data=Multi-game2025.06 | 93 | |
| SafeXArchitecture=Qwen3-30B-A3B2026.05 | 92.2 | |
| SFTBackbone=DeepSeek-Distill-Qwen-7B, Training Data=Multi-game2025.06 | 91.8 | |
| RTT-AONBackbone=Qwen3-4B-Instruct2026.04 | 91.8 | |
| MESAArchitecture=Qwen3-30B-A3B2026.05 | 91 | |
| DeepSeek-Distill-Qwen-7BBackbone=DeepSeek-Distill-Qwen-7B2025.06 | 90.8 | |
| Base(instruct)Architecture=Qwen3-30B-A3B2026.05 | 90.6 | |
| GRPOArchitecture=Qwen3-30B-A3B2026.05 | 90.2 | |
| Qwen3-4B-InstructBackbone=Qwen3-4B-Instruct2026.04 | 87.2 | |
| SPIRALBackbone=Qwen3-8B, Training Data=Multi-game2025.06 | 86.6 | |
| ConSteer-RLBackbone=Qwen3-8B-Base2026.06 | 86.6 | |
| GRPOBackbone=Qwen3-8B-Base2026.06 | 86 | |
| ConSteer-RLBackbone=Qwen3-4B-Base2026.06 | 85.8 | |
| Seq-ALPtemperature=1.0, sampling=average@32, multi-turn=true2026.03 | 84.29 | |
| Stair-SFTArchitecture=Qwen3-30B-A3B2026.05 | 83.8 | |
| GRPOBackbone=Qwen3-4B-Base2026.06 | 83.6 | |
| Token-ALPtemperature=1.0, sampling=average@32, multi-turn=true2026.03 | 83.48 | |
| Token-MIStemperature=1.0, sampling=average@32, multi-turn=true2026.03 | 83.29 | |
| Stair-DPOArchitecture=Qwen3-30B-A3B2026.05 | 83.2 | |
| SFTBackbone=Qwen3-8B, Training Data=Multi-game2025.06 | 82.8 | |
| Seq-Bypasstemperature=1.0, sampling=average@32, multi-turn=true2026.03 | 81.74 | |
| GRPOtemperature=1.0, sampling=average@32, multi-turn=true2026.03 | 80.9 | |
| Seq-MIStemperature=1.0, sampling=average@32, multi-turn=true2026.03 | 80.61 | |
| ConSteer-RLBackbone=Qwen2.5-Math-7B2026.06 | 78.4 | |
| SPIRALBackbone=Qwen3-4B, Training Data=Multi-game2025.06 | 78.2 | |
| GRPOBackbone=Qwen2.5-Math-7B2026.06 | 77.6 | |
| Qwen3-8B-BaseBackbone=Qwen3-8B2025.06 | 77 | |
| BaselineBackbone=Qwen3-8B-Base2026.06 | 76.6 | |
| SPIRALBackbone=Qwen3-4B, Training Data=Kuhn Poker2025.06 | 76.4 | |
| Qwen2.5-7B-InstructBackbone=Qwen2.5-7B-Instruct2026.04 | 74.8 | |
| SFTBackbone=Qwen3-4B, Training Data=Multi-game2025.06 | 74.2 | |
| SFTBackbone=Qwen3-4B, Training Data=Kuhn Poker2025.06 | 74 | |
| RTT-AONBackbone=Qwen2.5-7B-Instruct2026.04 | 73.8 | |
| Qwen3-4B-BaseBackbone=Qwen3-4B2025.06 | 73.4 | |
| BaselineBackbone=Qwen3-4B-Base2026.06 | 71.4 | |
| SFTArchitecture=Qwen3-30B-A3B2026.05 | 69.2 | |
| SPIRALBackbone=Octothinker-8B, Training Data=Multi-game2025.06 | 68.6 | |
| SFTBackbone=Octothinker-8B, Training Data=Multi-game2025.06 | 66 | |
| Octothinker-8B-BaseBackbone=Octothinker-8B2025.06 | 65.6 | |
| BaselineBackbone=Qwen2.5-Math-7B2026.06 | 65 | |
| SFTBackbone=Llama-3.1-8B-Instruct, Training Data=Multi-game2025.06 | 51.8 | |
| SPIRALBackbone=Llama-3.1-8B-Instruct, Training Data=Multi-game2025.06 | 49.8 | |
| Llama-3.1-8B-InstructBackbone=Llama-3.1-8B-Instruct2025.06 | 46.4 | |
| GRPO2026.05 | 43.67 | |
| GRPO + WeMask (TF)Setting=Training-free (TF)2026.05 | 43.47 | |
| GRPO + WeMask (TA)Setting=Training-aware (TA)2026.05 | 43.33 | |
| Qwen3-4B2026.05 | 43 | |
| RTT-AONBackbone=Llama3.2-3B-Instruct2026.04 | 40 | |
| Llama3.2-3B-InstructBackbone=Llama3.2-3B-Instruct2026.04 | 36.6 | |
| MESAArchitecture=DeepSeek-v2-Lite2026.05 | 28.8 | |
| GRPOArchitecture=DeepSeek-v2-Lite2026.05 | 28.2 | |
| Base(chat)Architecture=DeepSeek-v2-Lite2026.05 | 24.8 | |
| SafeXArchitecture=DeepSeek-v2-Lite2026.05 | 24.2 | |
| Stair-SFTArchitecture=DeepSeek-v2-Lite2026.05 | 16.4 | |
| SFTArchitecture=DeepSeek-v2-Lite2026.05 | 15 | |
| Stair-DPOArchitecture=DeepSeek-v2-Lite2026.05 | 14.4 |