Mathematical Reasoning on GSM8K (Acc, H100 TPS, TPF)
95.38GSM8K Accuracy (%)MIPRO
Evaluation results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| MIPROBackbone Model=Gemini-2-Flash, Prompt Optimization Technique=MIPRO2026.05 | 95.38 | — | — | — | — | |
| KVZipKV Cache Budget Sparsity=0.22025.10 | 95.2 | — | — | — | — | |
| BaselineBackbone Model=Gemini-2-Flash, Prompt Optimization Technique=Baseline2026.05 | 95.19 | — | — | — | — | |
| GEPABackbone Model=Gemini-2-Flash, Prompt Optimization Technique=GEPA2026.05 | 95.15 | — | — | — | — | |
| KVZipKV Cache Budget Sparsity=0.42025.10 | 95.1 | — | — | — | — | |
| DuoAttentionKV Cache Budget Sparsity=0.42025.10 | 95 | — | — | — | — | |
| PSAO_LLMBackbone Model=Gemini-2-Flash, Prompt Optimization Technique=PSAO_LLM2026.05 | 94.95 | — | — | — | — | |
| COPROBackbone Model=Gemini-2-Flash, Prompt Optimization Technique=COPRO2026.05 | 94.94 | — | — | — | — | |
| RLKVKV Cache Budget Sparsity=0.22025.10 | 94.9 | — | — | — | — | |
| DuoAttentionKV Cache Budget Sparsity=0.22025.10 | 94.8 | — | — | — | — | |
| ProTeGiBackbone Model=Gemini-2-Flash, Prompt Optimization Technique=ProTeGi2026.05 | 94.48 | — | — | — | — | |
| RLKVKV Cache Budget Sparsity=0.42025.10 | 94.4 | — | — | — | — | |
| PromptAgentBackbone Model=Gemini-2-Flash, Prompt Optimization Technique=PromptAgent2026.05 | 94.2 | — | — | — | — | |
| R-KVKV Cache Budget Sparsity=0.22025.10 | 94.2 | — | — | — | — | |
| SuCoModel Scale=7B2026.06 | 93.9 | — | — | — | — | |
| AdaptThinkModel Scale=7B2026.06 | 92.9 | — | — | — | — | |
| S-GRPOModel Scale=7B2026.06 | 92.8 | — | — | — | — | |
| LHRMsModel Scale=7B2026.06 | 92.4 | — | — | — | — | |
| KVZipKV Cache Budget Sparsity=0.62025.10 | 91.4 | — | — | — | — | |
| AdaCoTModel Scale=7B2026.06 | 91.4 | — | — | — | — | |
| RLKVKV Cache Budget Sparsity=0.62025.10 | 90.8 | — | — | — | — | |
| RLKVKV Cache Budget Sparsity=0.22025.10 | 90.7 | — | — | — | — | |
| RLKVKV Cache Budget Sparsity=0.42025.10 | 90.1 | — | — | — | — | |
| BaseAllowed Domains=—2026.05 | 89.6 | — | — | — | — | |
| DuoAttentionKV Cache Budget Sparsity=0.62025.10 | 89.6 | — | — | — | — | |
| DeepSeek-R1-DistillModel Scale=7B2026.06 | 89.3 | — | — | — | — | |
| DPO+FilterBase=Qwen 7b, Training Dataset=UltraFeedback, Pn Type=12025.10 | 89 | — | — | — | — | |
| DuoAttentionKV Cache Budget Sparsity=0.22025.10 | 88.9 | — | — | — | — | |
| VIGORBackbone=Qwen2.5-7B, Training Set=MATH2026.05 | 88.7 | — | — | — | — | |
| PALETTEAllowed Domains=Illegal | Violence2026.05 | 88.5 | — | — | — | — | |
| PALETTEAllowed Domains=Sexual | Illegal2026.05 | 88.2 | — | — | — | — | |
| KVZipKV Cache Budget Sparsity=0.22025.10 | 88.2 | — | — | — | — | |
| PALETTEAllowed Domains=Hate | Disinfo2026.05 | 88.1 | — | — | — | — | |
| DPO+FilterBase=Qwen 7b, Training Dataset=UltraFeedback, Pn Type=22025.10 | 88 | — | — | — | — | |
| DPO+FilterBase=Qwen 7b, Training Dataset=HelpSteer3, Pn Type=12025.10 | 88 | — | — | — | — | |
| R-KVKV Cache Budget Sparsity=0.22025.10 | 87.7 | — | — | — | — | |
| SuCoModel Scale=1.5B2026.06 | 87.7 | — | — | — | — | |
| DPO+FilterBase=Qwen 7b, Training Dataset=HelpSteer3, Pn Type=22025.10 | 87.5 | — | — | — | — | |
| PALETTEAllowed Domains=Disinfo | Sexual2026.05 | 87.3 | — | — | — | — | |
| INTUITORBackbone=Qwen2.5-7B, Training Set=MATH2026.05 | 87.19 | — | — | — | — | |
| BaseBase=Qwen 7b, Training Dataset=UltraFeedback, Pn Type=-2025.10 | 87 | — | — | — | — | |
| DPOBase=Qwen 7b, Training Dataset=UltraFeedback, Pn Type=22025.10 | 87 | — | — | — | — | |
| BaseBase=Qwen 7b, Training Dataset=HelpSteer3, Pn Type=-2025.10 | 87 | — | — | — | — | |
| DPOBase=Qwen 7b, Training Dataset=HelpSteer3, Pn Type=22025.10 | 87 | — | — | — | — | |
| Math-InstructModel Scale=7B2026.06 | 87 | — | — | — | — | |
| DPOBase=Qwen 7b, Training Dataset=UltraFeedback, Pn Type=12025.10 | 86.5 | — | — | — | — | |
| DPOBase=Qwen 7b, Training Dataset=HelpSteer3, Pn Type=12025.10 | 86.5 | — | — | — | — | |
| BaseAllowed Domains=None, Backbone=QWEN2.5-7B-INSTRUCT2026.05 | 86.5 | — | — | — | — | |
| PALETTEAllowed Domains=Violence | Hate, Backbone=QWEN2.5-7B-INSTRUCT2026.05 | 86.3 | — | — | — | — | |
| PALETTEAllowed Domains=Violence | Hate2026.05 | 86.2 | — | — | — | — | |
| PALETTEAllowed Domains=Disinfo | Sexual, Backbone=QWEN2.5-7B-INSTRUCT2026.05 | 85.7 | — | — | — | — | |
| LHRMsModel Scale=1.5B2026.06 | 85.7 | — | — | — | — | |
| PALETTEAllowed Domains=Illegal | Violence, Backbone=QWEN2.5-7B-INSTRUCT2026.05 | 85.6 | — | — | — | — | |
| R-KVKV Cache Budget Sparsity=0.42025.10 | 85.6 | — | — | — | — | |
| PALETTEAllowed Domains=Hate | Disinfo, Backbone=QWEN2.5-7B-INSTRUCT2026.05 | 85.4 | — | — | — | — | |
| KVZipKV Cache Budget Sparsity=0.42025.10 | 85 | — | — | — | — | |
| GT-RewardBackbone=Qwen2.5-7B, Training Set=MATH2026.05 | 84.8 | — | — | — | — | |
| PALETTEAllowed Domains=Sexual | Illegal, Backbone=QWEN2.5-7B-INSTRUCT2026.05 | 84.5 | — | — | — | — | |
| R-KVKV Cache Budget Sparsity=0.42025.10 | 84 | — | — | — | — | |
| CASTAllowed Domains=Disinfo | Sexual2026.05 | 83.8 | — | — | — | — | |
| CASTAllowed Domains=Hate | Disinfo2026.05 | 83.5 | — | — | — | — | |
| CASTAllowed Domains=Sexual | Illegal2026.05 | 83.4 | — | — | — | — | |
| S-GRPOModel Scale=1.5B2026.06 | 83.4 | — | — | — | — | |
| AdaptThinkModel Scale=1.5B2026.06 | 83.2 | — | — | — | — | |
| RLKVKV Cache Budget Sparsity=0.62025.10 | 83.1 | — | — | — | — | |
| AdaCoTModel Scale=1.5B2026.06 | 82.7 | — | — | — | — | |
| CASTAllowed Domains=Illegal | Violence2026.05 | 82.6 | — | — | — | — | |
| CASTAllowed Domains=Violence | Hate2026.05 | 82.2 | — | — | — | — | |
| DuoAttentionKV Cache Budget Sparsity=0.42025.10 | 82 | — | — | — | — | |
| GT-RewardBackbone=Qwen2.5-3B, Training Set=MATH2026.05 | 81.96 | — | — | — | — | |
| VIGORBackbone=Qwen2.5-3B, Training Set=MATH2026.05 | 81.8 | — | — | — | — | |
| wd1Generation Length=5122026.05 | 81.65 | — | — | — | — | |
| StableDRLGeneration Length=2562026.05 | 81.35 | — | — | — | — | |
| SPGGeneration Length=2562026.05 | 81.05 | — | — | — | — | |
| StableDRLGeneration Length=5122026.05 | 81.05 | — | — | — | — | |
| d1Generation Length=5122026.05 | 80.67 | — | — | — | — | |
| DeepSeek-R1-DistillModel Scale=1.5B2026.06 | 80.3 | — | — | — | — | |
| SPGGeneration Length=5122026.05 | 80.29 | — | — | — | — | |
| ESPOGeneration Length=2562026.05 | 80.21 | — | — | — | — | |
| CASTAllowed Domains=Illegal | Violence, Backbone=QWEN2.5-7B-INSTRUCT2026.05 | 80.1 | — | — | — | — | |
| Diffu-GRPOGeneration Length=5122026.05 | 79.98 | — | — | — | — | |
| ESPOGeneration Length=5122026.05 | 79.76 | — | — | — | — | |
| INTUITORBackbone=Qwen2.5-3B, Training Set=MATH2026.05 | 79.68 | — | — | — | — | |
| GDPOGeneration Length=5122026.05 | 79.23 | — | — | — | — | |
| Math-InstructModel Scale=1.5B2026.06 | 79 | — | — | — | — | |
| wd1Generation Length=2562026.05 | 78.85 | — | — | — | — | |
| CASTAllowed Domains=Sexual | Illegal, Backbone=QWEN2.5-7B-INSTRUCT2026.05 | 78.6 | — | — | — | — | |
| SFTGeneration Length=5122026.05 | 78.54 | — | — | — | — | |
| AdaBlock-dLLMGeneration Length=5122026.05 | 77.94 | — | — | — | — | |
| LLaDA-8B-InstructGeneration Length=5122026.05 | 77.63 | — | — | — | — | |
| CASTAllowed Domains=Hate | Disinfo, Backbone=QWEN2.5-7B-INSTRUCT2026.05 | 77.3 | — | — | — | — | |
| d1Generation Length=2562026.05 | 77.03 | — | — | — | — | |
| CASTAllowed Domains=Disinfo | Sexual, Backbone=QWEN2.5-7B-INSTRUCT2026.05 | 76.8 | — | — | — | — | |
| GDPOGeneration Length=2562026.05 | 76.57 | — | — | — | — | |
| CASTAllowed Domains=Violence | Hate, Backbone=QWEN2.5-7B-INSTRUCT2026.05 | 76.5 | — | — | — | — | |
| Diffu-GRPOGeneration Length=2562026.05 | 76.35 | — | — | — | — | |
| AdaBlock-dLLMGeneration Length=2562026.05 | 76.27 | — | — | — | — | |
| LLaDA-8B-InstructGeneration Length=2562026.05 | 76.19 | — | — | — | — | |
| SFTGeneration Length=2562026.05 | 75.59 | — | — | — | — | |
| Base (8B Instruct)Backbone=Llama 32026.05 | 75.4 | — | — | — | — |