Code Generation on MBPP (Accuracy, Thinking Token Usage (%))
71.2AccuracyDART
Evaluation results
| Method | Links | ||
|---|---|---|---|
| DARTModel Scale=32B, Thinking Strategy=DART2026.06 | 71.2 | 51 | |
| DARTModel Scale=8B, Thinking Strategy=DART2026.06 | 68.9 | 58 | |
| DARTModel Scale=14B, Thinking Strategy=DART2026.06 | 68.1 | 51 | |
| DARTModel Scale=4B, Thinking Strategy=DART2026.06 | 66.9 | 41 | |
| NTModel Scale=32B, Thinking Strategy=NT2026.06 | 65.8 | — | |
| ATModel Scale=32B, Thinking Strategy=AT2026.06 | 65.8 | — | |
| NTModel Scale=14B, Thinking Strategy=NT2026.06 | 64.6 | — | |
| ATModel Scale=14B, Thinking Strategy=AT2026.06 | 64.6 | — | |
| ATModel Scale=8B, Thinking Strategy=AT2026.06 | 64.2 | — | |
| ATModel Scale=4B, Thinking Strategy=AT2026.06 | 62.6 | — | |
| DARTModel Scale=1.7B, Thinking Strategy=DART2026.06 | 61.9 | 28 | |
| NTModel Scale=8B, Thinking Strategy=NT2026.06 | 60.7 | — | |
| ATModel Scale=1.7B, Thinking Strategy=AT2026.06 | 57.2 | — | |
| NTModel Scale=4B, Thinking Strategy=NT2026.06 | 56.4 | — | |
| DARTModel Scale=0.6B, Thinking Strategy=DART2026.06 | 51.4 | 16 | |
| ATModel Scale=0.6B, Thinking Strategy=AT2026.06 | 48.2 | — | |
| NTModel Scale=1.7B, Thinking Strategy=NT2026.06 | 48.2 | — | |
| NTModel Scale=0.6B, Thinking Strategy=NT2026.06 | 34.6 | — |