Mathematical Reasoning on GSM8K (Accuracy, Execution, Iteration Count)
88.34Accuracy (%)GPT-OSS-20B Cocktail FT
Evaluation results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| GPT-OSS-20B Cocktail FTModel=GPT-OSS-20B, Config=Cocktail FT, Model Scale=20B, Sig.=‡2026.03 | 88.34 | 93 | 60 | 70.8 | 1.52 | |
| GPT-OSS-20B Prolog FTModel=GPT-OSS-20B, Config=Prolog FT, Model Scale=20B, Sig.=†2026.03 | 86.12 | 91 | 55 | 69.2 | 1.61 | |
| Qwen2.5-Coder-32B Cocktail FTModel=Qwen2.5-Coder-32B, Config=Cocktail FT, Model Scale=32B, Sig.=‡2026.03 | 85.52 | 99.32 | 90.7 | 92.7 | 1.11 | |
| Qwen2.5-Coder-32B Prolog FTModel=Qwen2.5-Coder-32B, Config=Prolog FT, Model Scale=32B, Sig.=‡2026.03 | 85.14 | 99.24 | 98.7 | 41.2 | 1.02 | |
| GPT-OSS-20B BaseModel=GPT-OSS-20B, Config=Base, Model Scale=20B, Sig.=—2026.03 | 84.91 | 89.99 | 45 | 81.8 | 1.75 | |
| Qwen2.5-Coder-32B BaseModel=Qwen2.5-Coder-32B, Config=Base, Model Scale=32B, Sig.=—2026.03 | 80.29 | 97.04 | 96.4 | 17 | 1.07 | |
| Qwen3-8B BaseModel=Qwen3-8B, Config=Base, Model Scale=8B, Sig.=—2026.03 | 66.79 | 72.93 | 7.7 | 70.7 | 2.43 | |
| Qwen3-8B Cocktail FTModel=Qwen3-8B, Config=Cocktail FT, Model Scale=8B2026.03 | 64.51 | 77.1 | 69.8 | 24.1 | 1.54 | |
| Qwen3-8B Prolog FTModel=Qwen3-8B, Config=Prolog FT, Model Scale=8B, Sig.=†2026.03 | 63.15 | 75.96 | 69.2 | 27.7 | 1.6 | |
| Llama-3B-Instruct Cocktail FTModel=Llama-3B-Instruct, Config=Cocktail FT, Model Scale=3B, Sig.=†2026.03 | 27.07 | 57.92 | 53.8 | 8.9 | 1.89 | |
| Llama-3B-Instruct Prolog FTModel=Llama-3B-Instruct, Config=Prolog FT, Model Scale=3B2026.03 | 21.83 | 42.08 | 23.4 | 24.4 | 2.36 | |
| Llama-3B-Instruct BaseModel=Llama-3B-Instruct, Config=Base, Model Scale=3B, Sig.=—2026.03 | 21.53 | 66.49 | 47.5 | 36.2 | 1.9 |