# Attention Is (not) All You Need for Commonsense Reasoning Source: [Attention Is (not) All You Need for Commonsense Reasoning]() Collection index: [llms.txt]() Data updated: 2026-01-04T03:06:53.710Z Authors: Tassilo Klein, Moin Nabi Published: 2019-05-31T10:27:58.000Z ## Abstract The recently introduced BERT model exhibits strong performance on several language understanding benchmarks. In this paper, we describe a simple re-implementation of BERT for commonsense reasoning. We show that the attentions produced by BERT can be directly utilized for tasks such as the Pronoun Disambiguation Problem and Winograd Schema Challenge. Our proposed attention-guided commonsense reasoning method is conceptually simple yet empirically powerful. Experimental analysis on multiple datasets demonstrates that our proposed system performs remarkably well on all cases while outperforming the previously reported state of the art by a margin. While results suggest that BERT seems to implicitly learn to establish complex relationships between entities, solving commonsense reasoning tasks might require more than unsupervised models learned from huge text corpora. ## Recorded benchmark results A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. - [Pronoun Disambiguation on PDP-60](): Pronoun Disambiguation; PDP-60; Accuracy: 68.3. - [Pronoun Disambiguation on Winograd Schema Challenge](): Pronoun Disambiguation; Winograd Schema Challenge; Accuracy: 60.3. ## Original sources and implementation [Original paper]() [PDF]() [Implementation]()