Intent Classification on IntentGrasp Gem Set
4.4Performance (No Shuffle)Qwen3 4B
Evaluation results
| Method | Links | ||||||
|---|---|---|---|---|---|---|---|
| Qwen3 4BModel Family=Qwen3, Parameters=4B2026.05 | 4.4 | 6.29 | 6.56 | 7.28 | 6.13 | 1.06 | |
| Qwen3 8BModel Family=Qwen3, Parameters=8B2026.05 | 3.89 | 7.16 | 6.3 | — | 6.56 | 1.8 | |
| Llama3 8BModel Family=Llama3, Parameters=8B2026.05 | 3.33 | 4.33 | 4.15 | 5.14 | 4.24 | 0.64 | |
| Llama3 3BModel Family=Llama3, Parameters=3B2026.05 | 2.98 | 5.39 | 3.3 | 5.04 | 4.18 | 1.05 | |
| Qwen3 32BModel Family=Qwen3, Parameters=32B2026.05 | 1.28 | 5.69 | 3.83 | 4.5 | 3.83 | 1.61 | |
| Gemma4 E4BModel Family=Gemma4, Parameters=E4B2026.05 | 0.71 | 4.82 | 2.73 | 2.73 | 2.75 | 1.45 | |
| Gemma4 31BModel Family=Gemma4, Parameters=31B2026.05 | 0.5 | 3.75 | 3.54 | 3.99 | 2.95 | 1.42 | |
| Olmo3 32BModel Family=Olmo3, Parameters=32B2026.05 | 0.43 | 6.15 | 4 | 4.45 | 3.76 | 2.08 | |
| Gemma4 E2BModel Family=Gemma4, Parameters=E2B2026.05 | 0.25 | 4.36 | 3.44 | 3.94 | 3 | 1.61 | |
| Olmo3 7BModel Family=Olmo3, Parameters=7B2026.05 | 0.21 | 8.38 | 6.81 | 6.03 | 5.36 | 3.09 | |
| Llama3 70BModel Family=Llama3, Parameters=70B2026.05 | 0 | 5.95 | 5.17 | 5.26 | 4.1 | 2.38 |