Mathematical Reasoning on GSM8K (Accuracy, Avg Tokens, Avg Steps)
97.54AccuracyUpper Bound
Evaluation results
| Method | Links | ||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Upper BoundModel=Qwen3-8B, k=62025.08 | 97.54 | — | — | — | — | — | — | — | — | — | — | — | |
| ThoughtFoldModel=Qwen3-14B2026.06 | 96.8 | 894 | — | — | — | — | — | — | — | — | — | — | |
| Upper BoundModel=DS-Distill-Qwen-2.5-7B, k=62025.08 | 96.79 | — | — | — | — | — | — | — | — | — | — | — | |
| ThoughtFoldModel=DeepSeek-R1-Distill-Qwen-14B2026.06 | 96.7 | 701 | — | — | — | — | — | — | — | — | — | — | |
| S-GRPOModel=Qwen3-14B2026.06 | 96.3 | 952 | — | — | — | — | — | — | — | — | — | — | |
| SwiRBackbone=Qwen3-32B2026.04 | 96.21 | — | — | — | — | — | — | — | — | — | — | — | |
| S-GRPOModel=DeepSeek-R1-Distill-Qwen-14B2026.06 | 96.2 | 724 | — | — | — | — | — | — | — | — | — | — | |
| ThoughtFoldModel=Qwen3-8B2026.06 | 96.2 | 1,097 | — | — | — | — | — | — | — | — | — | — | |
| S-GRPOModel=Qwen3-8B2026.06 | 96.1 | 1,292 | — | — | — | — | — | — | — | — | — | — | |
| GRPOModel=Qwen3-14B2026.06 | 96.1 | 1,956 | — | — | — | — | — | — | — | — | — | — | |
| Short-RLModel=Qwen3-14B2026.06 | 96.1 | 1,034 | — | — | — | — | — | — | — | — | — | — | |
| SeLaRBackbone=Qwen3-32B2026.04 | 96.06 | — | — | — | — | — | — | — | — | — | — | — | |
| PiCSARModel=Qwen3-8B, k=62025.08 | 95.94 | — | — | — | — | — | — | — | — | — | — | — | |
| CoTBackbone=Qwen3-32B, Decoding Strategy=Greedy2026.04 | 95.91 | — | — | — | — | — | — | — | — | — | — | — | |
| SeLaRBackbone=Qwen3-8B2026.04 | 95.83 | — | — | — | — | — | — | — | — | — | — | — | |
| CoTBackbone=Qwen3-32B, Decoding Strategy=Sampling2026.04 | 95.83 | — | — | — | — | — | — | — | — | — | — | — | |
| GRPOModel=Qwen3-8B2026.06 | 95.8 | 2,355 | — | — | — | — | — | — | — | — | — | — | |
| RL + Length PenaltyModel=Qwen3-14B2026.06 | 95.8 | 1,090 | — | — | — | — | — | — | — | — | — | — | |
| Soft ThinkingBackbone=Qwen3-32B2026.04 | 95.75 | — | — | — | — | — | — | — | — | — | — | — | |
| STACKBackbone=DeepSeek-R1-Distill-Qwen-7B2026.04 | 95.7 | 563 | 17 | — | — | — | — | — | — | — | — | — | |
| Short-RLModel=Qwen3-8B2026.06 | 95.7 | 1,241 | — | — | — | — | — | — | — | — | — | — | |
| SwiRBackbone=Qwen3-8B2026.04 | 95.68 | — | — | — | — | — | — | — | — | — | — | — | |
| Self-ConsistencyModel=Qwen3-8B, k=62025.08 | 95.68 | — | — | — | — | — | — | — | — | — | — | — | |
| VanillaModel=Qwen3-14B2026.06 | 95.5 | 1,909 | — | — | — | — | — | — | — | — | — | — | |
| AverageModel=Qwen3-8B, k=62025.08 | 95.43 | — | — | — | — | — | — | — | — | — | — | — | |
| VanillaModel=Qwen3-8B2026.06 | 95.4 | 2,370 | — | — | — | — | — | — | — | — | — | — | |
| RL + Length PenaltyModel=Qwen3-8B2026.06 | 95.4 | 1,323 | — | — | — | — | — | — | — | — | — | — | |
| GRPOModel=DeepSeek-R1-Distill-Qwen-14B2026.06 | 95.3 | 2,120 | — | — | — | — | — | — | — | — | — | — | |
| CoTBackbone=Qwen3-8B, Decoding Strategy=Sampling2026.04 | 95.22 | — | — | — | — | — | — | — | — | — | — | — | |
| LatentMASBackbone=Qwen3-14B, Collaboration Method=Latent-based MAS, MAS Setting=Sequential2025.11 | 95.2 | 644 | — | — | — | — | 1,952 | — | — | — | — | — | |
| Qwen2.5-7B-Math-InsExecution Strategy=Prompting2026.02 | 95.2 | 323 | — | -3.88 | — | — | — | — | — | — | — | — | |
| OriginalBackbone=Qwen3-8B2025.10 | 95.15 | — | — | — | — | — | — | — | — | — | — | — | |
| CoTBackbone=Qwen3-8B, Decoding Strategy=Greedy2026.04 | 95.14 | — | — | — | — | — | — | — | — | — | — | — | |
| PromptBackbone=DeepSeek-R1-Distill-Qwen-7B2026.04 | 95.1 | 1,134 | 8.39 | — | — | — | — | — | — | — | — | — | |
| Short-RLModel=DeepSeek-R1-Distill-Qwen-14B2026.06 | 95.1 | 781 | — | — | — | — | — | — | — | — | — | — | |
| Soft ThinkingBackbone=Qwen3-8B2026.04 | 94.92 | — | — | — | — | — | — | — | — | — | — | — | |
| RL + Length PenaltyModel=DeepSeek-R1-Distill-Qwen-14B2026.06 | 94.7 | 775 | — | — | — | — | — | — | — | — | — | — | |
| DFTBackbone=Qwen3-8B2025.10 | 94.69 | — | — | — | — | — | — | — | — | — | — | — | |
| MuTISBackbone=DeepSeek-R1-Distill-Qwen-7B2026.04 | 94.6 | 614 | 15.41 | — | — | — | — | — | — | — | — | — | |
| ThoughtFoldModel=DeepSeek-R1-Distill-Qwen-7B2026.06 | 94.3 | 842 | — | — | — | — | — | — | — | — | — | — | |
| VanillaModel=DeepSeek-R1-Distill-Qwen-14B2026.06 | 94.2 | 2,129 | — | — | — | — | — | — | — | — | — | — | |
| VCOREBackbone=Qwen3-8B2025.10 | 94.16 | — | — | — | — | — | — | — | — | — | — | — | |
| LatentMASBackbone=Qwen3-8B, Collaboration Method=Latent-based MAS, MAS Setting=Sequential2025.11 | 93.8 | 860 | — | — | — | — | 543 | — | — | — | — | — | |
| TextMASBackbone=Qwen3-14B, Collaboration Method=Text-based MAS, MAS Setting=Sequential2025.11 | 93.8 | 3,324 | — | — | — | — | 3,729 | — | — | — | — | — | |
| S-GRPOModel=DeepSeek-R1-Distill-Qwen-7B2026.06 | 93.8 | 906 | — | — | — | — | — | — | — | — | — | — | |
| SFTBackbone=Qwen3-8B2025.10 | 93.63 | — | — | — | — | — | — | — | — | — | — | — | |
| RandomBackbone=Qwen3-8B2025.10 | 93.56 | — | — | — | — | — | — | — | — | — | — | — | |
| OriginalBackbone=DeepSeek-R1-Distill-Qwen-7B2026.04 | 93.5 | 1,478 | 6.32 | — | — | — | — | — | — | — | — | — | |
| GRPOModel=DeepSeek-R1-Distill-Qwen-7B2026.06 | 93.2 | 1,767 | — | — | — | — | — | — | — | — | — | — | |
| Qwen2.5-7B-MathExecution Strategy=Prompting2026.02 | 93.2 | 439 | — | -1.84 | — | — | — | — | — | — | — | — | |
| iw-SFTBackbone=Qwen3-8B2025.10 | 93.18 | — | — | — | — | — | — | — | — | — | — | — | |
| Short-RLModel=DeepSeek-R1-Distill-Qwen-7B2026.06 | 93.1 | 1,102 | — | — | — | — | — | — | — | — | — | — | |
| LCPOModel=DeepSeek-R1-Distill-Qwen-7B2025.08 | 92.95 | 796 | — | 1.14 | -52.53 | — | — | — | — | — | — | — | |
| Upper BoundModel=DS-Distill-llama-3-8B, k=62025.08 | 92.91 | — | — | — | — | — | — | — | — | — | — | — | |
| No DefenseModel=Qwen2.5-7B, Defense=No Defense2026.06 | 92.9 | — | — | — | — | — | — | — | — | — | — | — | |
| THRDModel=Qwen2.5-7B, Defense=Ours2026.06 | 92.6 | — | — | — | — | — | — | — | — | — | — | — | |
| ThinkSwitcherExecution Strategy=Offline, Base Model=R1-Distill-Qwen2.5-7B2026.02 | 92.5 | 1,389 | — | -0.35 | — | — | — | — | — | — | — | — | |
| VanillaModel=DeepSeek-R1-Distill-Qwen-7B2026.06 | 92.4 | 1,833 | — | — | — | — | — | — | — | — | — | — | |
| RL + Length PenaltyModel=DeepSeek-R1-Distill-Qwen-7B2026.06 | 92.4 | 1,062 | — | — | — | — | — | — | — | — | — | — | |
| ConCISEBackbone=DeepSeek-R1-Distill-Qwen-7B2026.04 | 92.3 | 773 | 11.94 | — | — | — | — | — | — | — | — | — | |
| TextMASBackbone=Qwen3-8B, Collaboration Method=Text-based MAS, MAS Setting=Sequential2025.11 | 92.3 | 2,324 | — | — | — | — | 1,739 | — | — | — | — | — | |
| ConCISE-SimPOExecution Strategy=Online, Base Model=R1-Distill-Qwen2.5-7B2026.02 | 92.1 | 715 | — | -0.71 | — | — | — | — | — | — | — | — | |
| OriginalModel=DeepSeek-R1-Distill-Qwen-7B2025.08 | 91.81 | 1,677 | — | — | — | — | — | — | — | — | — | — | |
| LatentMASBackbone=Qwen3-14B, MAS Setting=Hierarchical2025.11 | 91.6 | 495 | — | — | — | — | 1,631 | — | — | — | — | — | |
| Length-PenaltyExecution Strategy=Online, Base Model=R1-Distill-Qwen2.5-7B, Reproduced=true2026.02 | 91.6 | 931 | — | -0.27 | — | — | — | — | — | — | — | — | |
| CEEH-EAExecution Strategy=Online, Base Model=R1-Distill-Qwen2.5-7B2026.02 | 91.3 | 723 | — | -0.08 | — | — | — | — | — | — | — | — | |
| CEEH-MEExecution Strategy=Online, Base Model=R1-Distill-Qwen2.5-7B2026.02 | 91.3 | 646 | — | -0.08 | — | — | — | — | — | — | — | — | |
| AdaptThinkModel=DeepSeek-R1-Distill-Qwen-7B2025.08 | 91.28 | 354 | — | -0.53 | -78.89 | — | — | — | — | — | — | — | |
| R1-Distill-Qwen2.5-7BReproduced=true2026.02 | 91.2 | 1,479 | — | — | — | — | — | — | — | — | — | — | |
| Qwen2.5-7B-InsExecution Strategy=Prompting2026.02 | 90.9 | 279 | — | 0.3 | — | — | — | — | — | — | — | — | |
| TextMASBackbone=Qwen3-14B, MAS Setting=Hierarchical2025.11 | 90.8 | 3,021 | — | — | — | — | 3,675 | — | — | — | — | — | |
| DEERExecution Strategy=Offline, Base Model=R1-Distill-Qwen2.5-7B2026.02 | 90.6 | 917 | — | 0.41 | — | — | — | — | — | — | — | — | |
| SeLaRBackbone=Qwen3-1.7B2026.04 | 90.52 | — | — | — | — | — | — | — | — | — | — | — | |
| TextMASBackbone=Qwen3-8B, MAS Setting=Hierarchical2025.11 | 90.4 | 2,370 | — | — | — | — | 1,365 | — | — | — | — | — | |
| SwiRBackbone=Qwen3-1.7B2026.04 | 89.84 | — | — | — | — | — | — | — | — | — | — | — | |
| TextMASBackbone=Qwen3-4B, Collaboration Method=Text-based MAS, MAS Setting=Sequential2025.11 | 89.8 | 3,172 | — | — | — | — | 1,970 | — | — | — | — | — | |
| CoTBackbone=Qwen3-1.7B, Decoding Strategy=Sampling2026.04 | 89.61 | — | — | — | — | — | — | — | — | — | — | — | |
| Dynasor-CoTExecution Strategy=Offline, Base Model=R1-Distill-Qwen2.5-7B2026.02 | 89.6 | 1,285 | — | 0.64 | — | — | — | — | — | — | — | — | |
| Self-ConsistencyModel=DS-Distill-Qwen-2.5-7B, k=62025.08 | 89.5 | — | — | — | — | — | — | — | — | — | — | — | |
| LatentMASBackbone=Qwen3-8B, MAS Setting=Hierarchical2025.11 | 89.5 | 353 | — | — | — | — | 702 | — | — | — | — | — | |
| Soft ThinkingBackbone=Qwen3-1.7B2026.04 | 89.46 | — | — | — | — | — | — | — | — | — | — | — | |
| TextMASBackbone=Qwen3-4B, MAS Setting=Hierarchical2025.11 | 89.4 | 3,098 | — | — | — | — | 1,878 | — | — | — | — | — | |
| Switch (after Switch-GRPO)Reasoning style=hidden-state recurrence2026.06 | 89.2 | 1 | — | — | — | — | — | — | — | — | — | — | |
| Text-CoT (SFT)Reasoning style=explicit2026.06 | 88.6 | 1 | — | — | — | — | — | — | — | — | — | — | |
| LatentMASBackbone=Qwen3-4B, MAS Setting=Hierarchical2025.11 | 88.4 | 555 | — | — | — | — | 360 | — | — | — | — | — | |
| CoTBackbone=Qwen3-1.7B, Decoding Strategy=Greedy2026.04 | 88.32 | — | — | — | — | — | — | — | — | — | — | — | |
| LatentMASBackbone=Qwen3-4B, Collaboration Method=Latent-based MAS, MAS Setting=Sequential2025.11 | 88.2 | 607 | — | — | — | — | 375 | — | — | — | — | — | |
| PiCSARModel=DS-Distill-Qwen-2.5-7B, k=62025.08 | 88.18 | — | — | — | — | — | — | — | — | — | — | — | |
| LC-R1Execution Strategy=Online, Base Model=R1-Distill-Qwen2.5-7B2026.02 | 88.1 | 450 | — | 2.84 | — | — | — | — | — | — | — | — | |
| STACKBackbone=DeepSeek-R1-Distill-Qwen-1.5B2026.04 | 87.4 | 684 | 12.78 | — | — | — | — | — | — | — | — | — | |
| AverageModel=DS-Distill-Qwen-2.5-7B, k=62025.08 | 87.29 | — | — | — | — | — | — | — | — | — | — | — | |
| SpiritExecution Strategy=Online, Base Model=R1-Distill-Qwen2.5-7B2026.02 | 87.2 | 687 | — | 3.21 | — | — | — | — | — | — | — | — | |
| AutoThinkModel=DeepSeek-R1-Distill-Qwen-7B2025.08 | 86.81 | 809 | — | -5 | -51.76 | — | — | — | — | — | — | — | |
| DASTExecution Strategy=Online, Base Model=R1-Distill-Qwen2.5-7B2026.02 | 86.7 | 459 | — | 4.1 | — | — | — | — | — | — | — | — | |
| OriginalModel=DeepSeek-R1-Distill-Qwen-1.5B2025.08 | 86.28 | 2,457 | — | — | — | — | — | — | — | — | — | — | |
| LCPOModel=DeepSeek-R1-Distill-Qwen-1.5B2025.08 | 85.82 | 946 | — | -0.46 | -61.5 | — | — | — | — | — | — | — | |
| CEEH-EAExecution Strategy=Online, Base Model=R1-Distill-Qwen2.5-1.5B2026.02 | 85.8 | 997 | — | -1.47 | — | — | — | — | — | — | — | — | |
| CEEH-MEExecution Strategy=Online, Base Model=R1-Distill-Qwen2.5-1.5B2026.02 | 85.6 | 1,001 | — | -1.3 | — | — | — | — | — | — | — | — | |
| MuTISBackbone=DeepSeek-R1-Distill-Qwen-1.5B2026.04 | 85.4 | 895 | 9.54 | — | — | — | — | — | — | — | — | — | |
| PromptBackbone=DeepSeek-R1-Distill-Qwen-1.5B2026.04 | 85.3 | 2,134 | 4 | — | — | — | — | — | — | — | — | — |