# Lateral Thinking on BRAINTEASER Word Puzzle SemEval 2024 Task 9 (test) Source: [Lateral Thinking on BRAINTEASER Word Puzzle SemEval 2024 Task 9 (test)]() Collection index: [llms.txt]() Data updated: 2026-02-26T05:54:20.906Z Evaluation of model performance on the Lateral Thinking task using the BRAINTEASER Word Puzzle SemEval 2024 Task 9 test set, utilizing component-wise metrics like Original, Semantic, and Context scores. A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. | Scope | Recorded value | | --- | --- | | Task | Lateral Thinking | | Dataset (including any stated split) | BRAINTEASER Word Puzzle SemEval 2024 Task 9 (test) | | Metric | Original Score | | Unit | PERCENT | | Direction | Higher is better | | Results loaded | 7 of 7 | | Coverage | All results currently returned for this benchmark | Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol. ## Metrics - [Original Score](): Higher is better; unit: PERCENT. - [Semantic Score](): Higher is better; unit: PERCENT. - [Context Score](): Higher is better; unit: PERCENT. - [Original + Semantic Score](): Higher is better; unit: PERCENT. - [Overall Component Score (Original + Semantic + Context)](): Higher is better; unit: PERCENT. - [Overall Score](): Higher is better; unit: PERCENT. ## Recorded results | Method | Original Score | Result date | Report paper | | --- | --- | --- | --- | | Human | 91.7 | Not recorded | 68fa73471e209131cea5b7f9 | | DeBERTaV3 + AMMC + train-data-wp + Humor | 84.4 | Not recorded | 68fa73471e209131cea5b7f9 | | DeBERTaV3 + AMMC + train-data-wp+sp | 75 | Not recorded | 68fa73471e209131cea5b7f9 | | ChatGPT | 56.1 | Not recorded | 68fa73471e209131cea5b7f9 | | BERT-base + AMMC + train-data-wp+sp | 43.8 | Not recorded | 68fa73471e209131cea5b7f9 | | BERT-base + AMSC + train-data-wp+sp | 28.1 | Not recorded | 68fa73471e209131cea5b7f9 | | RoBERTa-L | 19.5 | Not recorded | 68fa73471e209131cea5b7f9 | ### Human Original Score: 91.7. Result ID: 699e99f15698665ec463144e. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: Not recorded. This record does not assert independent reproduction. ### DeBERTaV3 + AMMC + train-data-wp + Humor Original Score: 84.4. Result ID: 699e99f15698665ec4631454. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"DeBERTaV3","Architecture":"AutoModelForMultipleChoice (AMMC)","Training Data":"wp","Data Augmentation":"Humor"}. This record does not assert independent reproduction. ### DeBERTaV3 + AMMC + train-data-wp+sp Original Score: 75. Result ID: 699e99f15698665ec4631453. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Backbone":"DeBERTaV3","Architecture":"AutoModelForMultipleChoice (AMMC)","Training Data":"wp+sp"}. This record does not assert independent reproduction. ### ChatGPT Original Score: 56.1. Result ID: 699e99f15698665ec463144f. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: Not recorded. This record does not assert independent reproduction. ### BERT-base + AMMC + train-data-wp+sp Original Score: 43.8. Result ID: 699e99f15698665ec4631452. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Backbone":"BERT-base","Architecture":"AutoModelForMultipleChoice (AMMC)","Training Data":"wp+sp"}. This record does not assert independent reproduction. ### BERT-base + AMSC + train-data-wp+sp Original Score: 28.1. Result ID: 699e99f15698665ec4631451. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Backbone":"BERT-base","Architecture":"AutoModelForSequenceClassification (AMSC)","Training Data":"wp+sp"}. This record does not assert independent reproduction. ### RoBERTa-L Original Score: 19.5. Result ID: 699e99f15698665ec4631450. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"RoBERTa-Large"}. This record does not assert independent reproduction. Page 1 of 1. This is an index page, not the full collection.