Mathematical Reasoning on GSM8K (5-shot test)
95.27Strict Match AccuracyMarlin INT4
Evaluation results
| Method | Links | ||||||
|---|---|---|---|---|---|---|---|
| Marlin INT4TP=1, Throughput (tok/s)=73.6, Bits=4.02026.05 | 95.27 | 95.17 | — | — | — | — | |
| XFP V2 (auto)TP=1, Throughput (tok/s)=110, Bits=~3.972026.05 | 94.62 | 94.54 | — | — | — | — | |
| XFP V2 (auto)TP=2, Throughput (tok/s)=138, Bits=~3.972026.05 | 94.49 | 94.41 | — | — | — | — | |
| OPPOModel Backbone=Qwen2.5-7B, Evaluation Protocol=5-shot2025.09 | 82.79 | — | — | — | — | — | |
| TRLModel Backbone=Qwen2.5-7B, Evaluation Protocol=5-shot2025.09 | 82.56 | — | — | — | — | — | |
| Prefix-CacheGen Length=256, Evaluation Protocol=5-shot2026.01 | 79.5 | — | — | 9 | 3.9 | — | |
| ProphetBackbone=LLaDA-8B-Instruct, KV Cache=false, Generation Length=2562026.05 | 79.4 | — | — | — | 1.6 | 11 | |
| TSPDBackbone=LLaDA-8B-Instruct, KV Cache=false, Generation Length=2562026.05 | 79.4 | — | — | — | 5 | 34.2 | |
| LLaDAGen Length=256, Evaluation Protocol=5-shot2026.01 | 79.3 | — | — | 35.5 | 1 | — | |
| VanillaBackbone=LLaDA-8B-Instruct, KV Cache=true, Generation Length=2562026.05 | 79.3 | — | — | — | 1 | 6.9 | |
| Learn2PDBackbone=LLaDA-8B-Instruct, KV Cache=false, Generation Length=2562026.05 | 79.1 | — | — | — | 4.2 | 29 | |
| TSPDBackbone=LLaDA-8B-Instruct, KV Cache=true, Generation Length=2562026.05 | 78.8 | — | — | — | 11.2 | 77.3 | |
| Streaming-dLLMGen Length=512, Evaluation Protocol=5-shot2026.01 | 78.7 | — | — | 2.3 | 36.6 | — | |
| Credit DecodingBackbone=LLaDA-8B-Instruct, KV Cache=true, Generation Length=2562026.05 | 78.7 | — | — | — | 7.9 | 54.5 | |
| Fast-dLLMGen Length=256, Evaluation Protocol=5-shot2026.01 | 78.5 | — | — | 4.6 | 7.7 | — | |
| Streaming-dLLMGen Length=256, Evaluation Protocol=5-shot2026.01 | 78.5 | — | — | 2.2 | 16.1 | — | |
| Fast-dLLMBackbone=LLaDA-8B-Instruct, KV Cache=true, Generation Length=2562026.05 | 78.5 | — | — | — | 7.8 | 53.8 | |
| dKV-CacheGen Length=512, Evaluation Protocol=5-shot2026.01 | 77.6 | — | — | 49 | 1.7 | — | |
| LLaDAGen Length=512, Evaluation Protocol=5-shot2026.01 | 77.5 | — | — | 84.1 | 1 | — | |
| Fast-dLLMGen Length=512, Evaluation Protocol=5-shot2026.01 | 77.2 | — | — | 8.5 | 9.9 | — | |
| Prefix-CacheGen Length=512, Evaluation Protocol=5-shot2026.01 | 77 | — | — | 24.7 | 3.4 | — | |
| dKV-CacheGen Length=256, Evaluation Protocol=5-shot2026.01 | 76.5 | — | — | 20.2 | 1.8 | — | |
| LLaDABackbone=LLaDA-8B-Base, Speedup=1.0×2026.03 | 71.11 | — | — | — | — | 9.2 | |
| Uniform Regularized LoopingModel=Gemma, Size=9B, Shots=5-shot2026.02 | 68.52 | — | — | — | — | — | |
| BaselineModel=Gemma, Size=9B, Shots=5-shot2026.02 | 68.46 | — | — | — | — | — | |
| Moving Average Regularized LoopingModel=Gemma, Size=9B, Shots=5-shot2026.02 | 68.39 | — | — | — | — | — | |
| Noise AblationModel=Gemma, Size=9B, Shots=5-shot2026.02 | 68.33 | — | — | — | — | — | |
| Auto-Align Regularized LoopingModel=Gemma, Size=9B, Shots=5-shot2026.02 | 68.31 | — | — | — | — | — | |
| ES-dLLMBackbone=LLaDA-8B-Base, Speedup=15.3×2026.03 | 67.63 | — | — | — | — | 140.45 | |
| DualCacheBackbone=LLaDA-8B-Base, Speedup=12.7×2026.03 | 66.19 | — | — | — | — | 116.46 | |
| OPPOModel Backbone=Qwen2.5-3B, Evaluation Protocol=5-shot2025.09 | 63.53 | — | — | — | — | — | |
| TRLModel Backbone=Qwen2.5-3B, Evaluation Protocol=5-shot2025.09 | 63.46 | — | — | — | — | — | |
| GradientModel=Gemma2-2b, Fine-tuning Dataset=MetaMathQA, Trainable Parameters (%)=0.97%, Evaluation Protocol=5-shot2024.12 | 37.15 | 50.27 | 43.71 | — | — | — | |
| GRASPModel=Gemma2-2b, Fine-tuning Dataset=MetaMathQA, Trainable Parameters (%)=0.97%, Evaluation Protocol=5-shot2024.12 | 37.03 | 50.15 | 43.59 | — | — | — | |
| Taylor-FOModel=Gemma2-2b, Fine-tuning Dataset=MetaMathQA, Trainable Parameters (%)=0.97%, Evaluation Protocol=5-shot2024.12 | 30.25 | 40.33 | 35.29 | — | — | — | |
| Fisher-InfoModel=Gemma2-2b, Fine-tuning Dataset=MetaMathQA, Trainable Parameters (%)=0.97%, Evaluation Protocol=5-shot2024.12 | 30.25 | 41.47 | 35.86 | — | — | — | |
| FORCEModel=Gemma2-2b, Fine-tuning Dataset=MetaMathQA, Trainable Parameters (%)=0.97%, Evaluation Protocol=5-shot2024.12 | 29.95 | 39.88 | 34.91 | — | — | — | |
| SNIPModel=Gemma2-2b, Fine-tuning Dataset=MetaMathQA, Trainable Parameters (%)=0.97%, Evaluation Protocol=5-shot2024.12 | 29.64 | 39.8 | 34.72 | — | — | — | |
| LoRAModel=Gemma2-2b, Fine-tuning Dataset=MetaMathQA, Trainable Parameters (%)=0.97%, Evaluation Protocol=5-shot2024.12 | 28.81 | 39.2 | 34 | — | — | — | |
| SynFlowModel=Gemma2-2b, Fine-tuning Dataset=MetaMathQA, Trainable Parameters (%)=0.97%, Evaluation Protocol=5-shot2024.12 | 27.75 | 37.76 | 32.75 | — | — | — | |
| MagnitudeModel=Gemma2-2b, Fine-tuning Dataset=MetaMathQA, Trainable Parameters (%)=0.97%, Evaluation Protocol=5-shot2024.12 | 27.07 | 37.45 | 32.26 | — | — | — | |
| Moving Average Regularized LoopingModel=Gemma, Size=2B, Shots=5-shot2026.02 | 26.38 | — | — | — | — | — | |
| Auto-Align Regularized LoopingModel=Gemma, Size=2B, Shots=5-shot2026.02 | 26.08 | — | — | — | — | — | |
| Uniform Regularized LoopingModel=Gemma, Size=2B, Shots=5-shot2026.02 | 26 | — | — | — | — | — | |
| BaselineModel=Gemma, Size=2B, Shots=5-shot2026.02 | 25.02 | — | — | — | — | — | |
| Noise AblationModel=Gemma, Size=2B, Shots=5-shot2026.02 | 24.79 | — | — | — | — | — | |
| PretrainedModel=Gemma2-2b, Trainable Parameters (%)=0%, Evaluation Protocol=5-shot2024.12 | 17.66 | 24.56 | 21.11 | — | — | — |