Intent Classification on Banking77 (dev)
100CoverageLLM prompting
Evaluation results
| Method | Links | ||||||
|---|---|---|---|---|---|---|---|
| LLM promptingEvaluation protocol=Zero-shot prompting2026.07 | 100 | — | — | 98 | 98 | 98 | |
| Zero-shot NLI (DeBERTa-v3)Backbone=DeBERTa-v3, Evaluation protocol=Zero-shot NLI2026.07 | 100 | — | — | 96.5 | 96.5 | 96.5 | |
| LogReg (MiniLM emb.)Classifier=Logistic Regression, Embedding=MiniLM2026.07 | 100 | — | — | 95 | 95 | 95 | |
| RuleChefEvaluation protocol=few-shot2026.07 | 62.5 | 126 | — | 97.6 | 61 | 75.1 | |
| RuleChefEvaluation protocol=Observation mode (no labels), Stage=after 50 calls2026.07 | 49.5 | 40 | 48 | 96 | 49.5 | 63.6 | |
| RuleChefEvaluation protocol=Observation mode (no labels), Stage=after 25 calls2026.07 | 42.5 | 24 | 41 | 95.3 | 40.5 | 56.8 | |
| RuleChefEvaluation protocol=Observation mode (no labels), Stage=after 10 calls2026.07 | 20.5 | 14 | 19.5 | 92.7 | 19 | 31.5 |