Common Sense Reasoning on Winograd Schema Challenge 273 sentences (original)
61.5AccuracyEnsemble of 10 Unsupervised LMs-partial
Evaluation results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| Ensemble of 10 Unsupervised LMs-partialEnsemble Size=10, Scoring Strategy=partial, Model Type=Unsupervised LMs2018.06 | 61.5 | — | — | — | — | |
| Word-LM-partialGranularity=Word, Scoring Strategy=partial2018.06 | 56.4 | — | — | — | — | |
| Word-LM-fullGranularity=Word, Scoring Strategy=full2018.06 | 53.8 | — | — | — | — | |
| USSM + Supervised DeepNet + Knowledge Basecomponents=USSM, Supervised DeepNet, Knowledge Base2018.06 | 52.8 | — | — | — | — | |
| USSM + Knowledge Basecomponents=USSM, Knowledge Base2018.06 | 52 | — | — | — | — | |
| Char-LM-partialGranularity=Character, Scoring Strategy=partial2018.06 | 51.3 | — | — | — | — | |
| Char-LM-fullGranularity=Character, Scoring Strategy=full2018.06 | 51.3 | — | — | — | — | |
| Random guess2018.06 | 50 | — | — | — | — | |
| AGQManual information=false, Query generation=automatic, Semantic similarity filter=false2018.10 | — | 77 | 56 | 28 | 38 | |
| AGQ+FManual information=false, Query generation=automatic, Semantic similarity filter=true2018.10 | — | 80 | 63 | 29 | 40 | |
| AGQSManual information=false, Query generation=automatic (with synonyms), Semantic similarity filter=false2018.10 | — | 114 | 57 | 42 | 48 | |
| AGQS+FManual information=false, Query generation=automatic (with synonyms), Semantic similarity filter=true2018.10 | — | 119 | 60 | 44 | 51 | |
| L2017Manual information=true2018.10 | — | 43 | 61 | 15 | 25 | |
| MGQManual information=true, Query generation=manual2018.10 | — | 118 | 60 | 43 | 50 | |
| S2015Manual information=false2018.10 | — | 49 | 92 | 18 | 30 |