Code Generation on HumanEval (Accuracy and Thinking Token Usage)
95.1AccuracyDART
Evaluation results
| Method | Links | ||
|---|---|---|---|
| DARTModel Scale=32B, Thinking Strategy=DART2026.06 | 95.1 | 63 | |
| DARTModel Scale=4B, Thinking Strategy=DART2026.06 | 91.5 | 58 | |
| DARTModel Scale=1.7B, Thinking Strategy=DART2026.06 | 86.6 | 40 | |
| ATModel Scale=1.7B, Thinking Strategy=AT2026.06 | 85.4 | — | |
| NTModel Scale=32B, Thinking Strategy=NT2026.06 | 79.9 | — | |
| DARTModel Scale=8B, Thinking Strategy=DART2026.06 | 78.7 | 55 | |
| DARTModel Scale=14B, Thinking Strategy=DART2026.06 | 78.7 | 60 | |
| ATModel Scale=4B, Thinking Strategy=AT2026.06 | 74.4 | — | |
| ATModel Scale=32B, Thinking Strategy=AT2026.06 | 72.6 | — | |
| NTModel Scale=14B, Thinking Strategy=NT2026.06 | 71.3 | — | |
| NTModel Scale=4B, Thinking Strategy=NT2026.06 | 67.1 | — | |
| ATModel Scale=14B, Thinking Strategy=AT2026.06 | 66.5 | — | |
| NTModel Scale=8B, Thinking Strategy=NT2026.06 | 60.4 | — | |
| ATModel Scale=8B, Thinking Strategy=AT2026.06 | 59.1 | — | |
| NTModel Scale=1.7B, Thinking Strategy=NT2026.06 | 55.5 | — | |
| DARTModel Scale=0.6B, Thinking Strategy=DART2026.06 | 50.6 | 19 | |
| ATModel Scale=0.6B, Thinking Strategy=AT2026.06 | 48.8 | — | |
| NTModel Scale=0.6B, Thinking Strategy=NT2026.06 | 27.4 | — |