# Augmenting Lateral Thinking in Language Models with Humor and Riddle Data for the BRAINTEASER Task

Source: [Augmenting Lateral Thinking in Language Models with Humor and Riddle Data for the BRAINTEASER Task](<https://www.sota2.com/research/paper/68fa73471e209131cea5b7f9>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2026-02-26T05:00:04.574Z

Authors: Mina Ghashami, Soumya Smruti Mishra

Published: 2024-05-16T18:26:38.000Z

## Abstract

The SemEval 2024 BRAINTEASER task challenges language models to perform lateral thinking -- a form of creative, non-linear reasoning that remains underexplored in NLP. The task comprises two subtasks, Sentence Puzzle and Word Puzzle, requiring models to defy conventional commonsense associations. We present a system that fine-tunes DeBERTaV3 using HuggingFace's AutoModelForMultipleChoice architecture. We augment the provided training data with two additional sources: (1) a humor-style question-answering dataset generated via GPT-4 prompting, and (2) the RiddleSense dataset. This data augmentation strategy is motivated by the observation that humor and riddles share the lateral reasoning structure required by the task. Our best system achieves 92.5\\% overall accuracy on the Sentence Puzzle subtask and 80.2\\% on the Word Puzzle subtask, ranking 6th out of 31 teams and 10th out of 23 teams, respectively. We further show that the choice of task formulation matters: framing the problem as multiple-choice rather than sequence classification yields a 10-point accuracy improvement with the same base model. Our analysis reveals that data augmentation with humor and riddle data is particularly effective for sentence-level lateral reasoning, while word-level puzzles remain a harder challenge.

## Recorded benchmark results

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

- [Lateral Thinking on BRAINTEASER Sentence Puzzle SemEval 2024 Task 9 (test)](<https://www.sota2.com/research/sota/lateral-thinking-on-brainteaser-sentence-puzzle-semeval-2024-task-9-test.md>): Lateral Thinking; BRAINTEASER Sentence Puzzle SemEval 2024 Task 9 (test); Original Score: 92.5.

- [Lateral Thinking on BRAINTEASER Word Puzzle SemEval 2024 Task 9 (test)](<https://www.sota2.com/research/sota/lateral-thinking-on-brainteaser-word-puzzle-semeval-2024-task-9-test.md>): Lateral Thinking; BRAINTEASER Word Puzzle SemEval 2024 Task 9 (test); Original Score: 84.4.

## Original sources and implementation

[Original paper](<https://arxiv.org/abs/2405.10385>)

[PDF](<https://arxiv.org/pdf/2405.10385.pdf>)