Suboptimal-claim detection on ClaimRev (test)
67.1AccuracyFT-DeBERTa
Evaluation results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| FT-DeBERTaApproach=Fine-tuned2023.05 | 67.1 | 66.6 | 71.3 | 55.9 | 62.6 | |
| FT-ELECTRAApproach=Fine-tuned2023.05 | 63.8 | 62.9 | 68.8 | 49 | 57.2 | |
| ELECTRA + SVMApproach=Embed. + SVM, Embedding=ELECTRA2023.05 | 63.2 | 62.9 | 65.1 | 55 | 59.6 | |
| FT-BERTApproach=Fine-tuned2023.05 | 63.1 | 61.7 | 70.1 | 44.2 | 54.2 | |
| BERT + SVMApproach=Embed. + SVM, Embedding=BERT2023.05 | 62.1 | 61.8 | 63.5 | 54.7 | 58.8 | |
| DeBERTa + SVMApproach=Embed. + SVM, Embedding=DeBERTa2023.05 | 61.5 | 61.2 | 63.2 | 52.9 | 57.6 | |
| Flair + SVMApproach=Embed. + SVM, Embedding=Flair2023.05 | 60.1 | 60.1 | 60.2 | 56.9 | 58.5 | |
| Glove + SVMApproach=Embed. + SVM, Embedding=Glove2023.05 | 54.9 | 54.9 | 54.9 | 50 | 52.1 | |
| Random baseline2023.05 | 50 | 50 | 50 | 50 | 50 |