Arithmetic Calculation on DeepMind-Math (Acc, MAE, R2)
71.96AccuracySMMD
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| SMMDModel=Llama3, Params=8B2026.06 | 71.96 | 1.16 | 0.92 | |
| CEModel=Llama3, Params=8B2026.06 | 70.04 | 1.28 | 0.91 | |
| GCEModel=Llama3, Params=8B2026.06 | 69.59 | 1.19 | 0.91 | |
| NTILModel=Llama3, Params=8B2026.06 | 67.91 | 1.66 | 0.85 | |
| NTLModel=Llama3, Params=8B2026.06 | 67.01 | 1.66 | 0.86 | |
| SMMDModel=SmolLM3, Params=3B2026.06 | 64.06 | 1.88 | 0.82 | |
| CEModel=SmolLM3, Params=3B2026.06 | 61.94 | 2.07 | 0.8 | |
| NTILModel=SmolLM3, Params=3B2026.06 | 61.28 | 2.14 | 0.81 | |
| GCEModel=SmolLM3, Params=3B2026.06 | 60.45 | 2.21 | 0.8 | |
| NTLModel=SmolLM3, Params=3B2026.06 | 60.02 | 2.36 | 0.79 | |
| SMMDModel=Qwen2.5, Params=1.5B2026.06 | 53.83 | 2.66 | 0.77 | |
| NTILModel=Qwen2.5, Params=1.5B2026.06 | 52.99 | 2.6 | 0.75 | |
| GCEModel=Qwen2.5, Params=1.5B2026.06 | 52.36 | 2.54 | 0.78 | |
| NTLModel=Qwen2.5, Params=1.5B2026.06 | 52.05 | 2.92 | 0.71 | |
| CEModel=Qwen2.5, Params=1.5B2026.06 | 51.83 | 2.62 | 0.76 | |
| SMMDModel=Qwen2.5, Params=0.5B2026.06 | 45.93 | 3.25 | 0.69 | |
| NTLModel=Qwen2.5, Params=0.5B2026.06 | 44.79 | 3.56 | 0.67 | |
| CEModel=Qwen2.5, Params=0.5B2026.06 | 43.39 | 3.38 | 0.67 | |
| NTILModel=Qwen2.5, Params=0.5B2026.06 | 43.03 | 3.76 | 0.64 | |
| GCEModel=Qwen2.5, Params=0.5B2026.06 | 42.04 | 3.32 | 0.73 |