Aggregate Outcome Classification on CUE-Bench six tasks combined
10.1NB (%)Qwen2.5-0.5B
Evaluation Results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| Qwen2.5-0.5BParameter Count=0.5B, Series=Qwen2.52026.06 | 10.1 | 19.9 | 29.5 | 1 | 49.7 | |
| Qwen2.5-Coder-0.5BParameter Count=0.5B, Series=Qwen2.5-Coder2026.06 | 10.1 | 18.9 | 33.3 | 3.5 | 44.3 | |
| Qwen3-0.6BParameter Count=0.6B, Series=Qwen32026.06 | 7.2 | 25.5 | 30.7 | 3.5 | 40.3 | |
| Qwen2.5-1.5BParameter Count=1.5B, Series=Qwen2.52026.06 | 6.8 | 20.4 | 25.7 | 4.5 | 49.4 | |
| Llama-2-7BParameter Count=7B, Series=Llama-22026.06 | 6.1 | 21.6 | 41.1 | 3 | 34.2 | |
| Qwen2.5-Coder-1.5BParameter Count=1.5B, Series=Qwen2.5-Coder2026.06 | 5.4 | 25.7 | 22.1 | 6.5 | 45.7 | |
| Qwen2.5-3BParameter Count=3B, Series=Qwen2.52026.06 | 5.3 | 30.9 | 29.7 | 2.6 | 36.8 | |
| Qwen3-1.7BParameter Count=1.7B, Series=Qwen32026.06 | 4.9 | 27.2 | 31.7 | 3.6 | 37.6 | |
| CodeLlama-7BParameter Count=7B, Series=CodeLlama2026.06 | 4.7 | 24.1 | 22.5 | 11.1 | 42.2 | |
| Qwen2.5-Coder-3BParameter Count=3B, Series=Qwen2.5-Coder2026.06 | 3.9 | 36.2 | 34.1 | 3.1 | 26.6 | |
| Qwen2.5-7BParameter Count=7B, Series=Qwen2.52026.06 | 3.8 | 36.7 | 31.7 | 3.2 | 28.4 | |
| DeepSeek-Coder-6.7BParameter Count=6.7B, Series=DeepSeek-Coder2026.06 | 3.2 | 38.4 | 34.7 | 6.4 | 20.5 | |
| Qwen2.5-Coder-7BParameter Count=7B, Series=Qwen2.5-Coder2026.06 | 2.7 | 41.5 | 26.4 | 8.5 | 23.7 | |
| Qwen3-4BParameter Count=4B, Series=Qwen32026.06 | 2.5 | 40.7 | 27.5 | 5.2 | 26.6 | |
| Qwen2.5-Coder-14BParameter Count=14B, Series=Qwen2.5-Coder2026.06 | 1.4 | 50.3 | 31.1 | 4.7 | 14 | |
| Qwen3-8BParameter Count=8B, Series=Qwen32026.06 | 1.1 | 44.2 | 31.6 | 4 | 20.3 |