Lateral Thinking on BRAINTEASER Word Puzzle SemEval 2024 Task 9 (test)
91.7Original ScoreHuman
Evaluation results
| Method | Links | ||||||
|---|---|---|---|---|---|---|---|
| Human2024.05 | 91.7 | 91.7 | 91.7 | 91.7 | 90 | 91.7 | |
| DeBERTaV3 + AMMC + train-data-wp + HumorBackbone=DeBERTaV3, Architecture=AutoModelForMultipleChoice (AMMC), Training Data=wp, Data Augmentation=Humor2024.05 | 84.4 | 81.2 | 75 | 78.1 | 59.4 | 80.2 | |
| DeBERTaV3 + AMMC + train-data-wp+spBackbone=DeBERTaV3, Architecture=AutoModelForMultipleChoice (AMMC), Training Data=wp+sp2024.05 | 75 | 75 | 62.5 | 71.9 | 50 | 70.8 | |
| ChatGPT2024.05 | 56.1 | 52.4 | 51.8 | 43.9 | 29.2 | 53.5 | |
| BERT-base + AMMC + train-data-wp+spBackbone=BERT-base, Architecture=AutoModelForMultipleChoice (AMMC), Training Data=wp+sp2024.05 | 43.8 | 37.5 | 40.6 | 34.4 | 37.5 | 40.6 | |
| BERT-base + AMSC + train-data-wp+spBackbone=BERT-base, Architecture=AutoModelForSequenceClassification (AMSC), Training Data=wp+sp2024.05 | 28.1 | 31.2 | 37.5 | 3.1 | 0 | 32.3 | |
| RoBERTa-LBackbone=RoBERTa-Large2024.05 | 19.5 | 19.5 | 23.2 | 14.6 | 6.1 | 20.7 |