Intent Detection on Comment Dataset Out-of-Distribution
65.6F1 Score (val)BOT-MOD
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| BOT-MODUser Model=Mistral-7B2026.05 | 65.6 | 73.46 | 50.41 | |
| zero-shot+User Model=Mistral-7B2026.05 | 63.06 | 70.89 | 47.98 | |
| CoTUser Model=Mistral-7B2026.05 | 59.77 | 60.49 | 58.13 | |
| BOT-MODUser Model=Llama-3.12026.05 | 58.75 | 62.57 | 50.78 | |
| BOT-MODUser Model=Qwen32026.05 | 58.71 | 62.95 | 49.9 | |
| zero-shotUser Model=Mistral-7B2026.05 | 57.26 | 65.08 | 42.47 | |
| self-consistencyUser Model=Mistral-7B2026.05 | 56.7 | 63.88 | 42.94 | |
| zero-shot+User Model=Llama-3.12026.05 | 54.13 | 53.34 | 56.05 | |
| self-consistencyUser Model=Qwen32026.05 | 53.34 | 53.95 | 51.93 | |
| zero-shotUser Model=Qwen32026.05 | 53.04 | 53.9 | 51.09 | |
| self-consistencyUser Model=Llama-3.12026.05 | 52.82 | 53.38 | 51.56 | |
| zero-shotUser Model=Llama-3.12026.05 | 52.54 | 52.97 | 51.56 | |
| zero-shot+User Model=Qwen32026.05 | 52.52 | 57.04 | 43.3 | |
| self-refineUser Model=Mistral-7B2026.05 | 49.72 | 55.9 | 38.65 | |
| BERTUser Model=Mistral-7B2026.05 | 48.86 | 53.73 | 39.15 | |
| CoTUser Model=Qwen32026.05 | 47.19 | 46.69 | 48.4 | |
| self-refineUser Model=Llama-3.12026.05 | 47.17 | 50.69 | 40.25 | |
| BERTUser Model=Qwen32026.05 | 46.91 | 51.23 | 38.18 | |
| CoTUser Model=Llama-3.12026.05 | 46.07 | 46.54 | 45.04 | |
| BERTUser Model=Llama-3.12026.05 | 44.16 | 46.89 | 38.4 | |
| self-refineUser Model=Qwen32026.05 | 41.64 | 49.1 | 28.44 |