List your company
ResearchFormerly WizWand

Intent Classification on BANKING77 (10-shot)

87.95AccuracyDynamic Label Name Refinement
72.017276.153680.2984.4264Jun 8, 2023Sep 9, 2023Dec 12, 2023Mar 14, 2024Jun 16, 2024Sep 17, 2024Dec 20, 2024
Updated 8mo ago

Evaluation results

MethodLinks
Dynamic Label Name RefinementBackbone=Llama3-8b-inst, Shot Count=10-shot2024.1287.95
Dynamic Label Name RefinementBackbone=Qwen2.5-7b-inst, Shot Count=10-shot2024.1287.3
Chain-of-ThoughtBackbone=Qwen2.5-7b-inst, Shot Count=10-shot2024.1287.05
DFT++ (w/ CA, SSD)Backbone=RoBERTa, Context Augmentation=true, Sequential Self-Distillation=true2023.0686.14
DFT++ (w/ CA)Backbone=RoBERTa, Context Augmentation=true2023.0685.95
RawBackbone=Llama3-8b-inst, Shot Count=10-shot2024.1285.88
RawBackbone=Qwen2.5-7b-inst, Shot Count=10-shot2024.1285.68
Chain-of-ThoughtBackbone=Llama3-8b-inst, Shot Count=10-shot2024.1285.48
DFT++ (w/ SSD)Backbone=RoBERTa, Sequential Self-Distillation=true2023.0684.95
DFTBackbone=RoBERTa2023.0684.77
SE-NLIBackbone=RoBERTa2023.0684.65
IsoIntentRoBERTaBackbone=RoBERTa2023.0684.49
IntentRoBERTaBackbone=RoBERTa2023.0683.94
SE-ParaphraseBackbone=RoBERTa2023.0682.85
DNNC-NLIBackbone=RoBERTa2023.0679.51
CPFTBackbone=RoBERTa2023.0678.57
DNNC-IntentBackbone=RoBERTa2023.0677.69
Dynamic Label Name RefinementBackbone=Qwen2.5-1.5b-inst, Shot Count=10-shot2024.1277.34
RawBackbone=Qwen2.5-1.5b-inst, Shot Count=10-shot2024.1273.31
Chain-of-ThoughtBackbone=Qwen2.5-1.5b-inst, Shot Count=10-shot2024.1272.63