Fake News Stance Detection on FNC 1 (test)
74.09AgreeCNN + biLSTM
Evaluation results
| Method | Links | |||||||
|---|---|---|---|---|---|---|---|---|
| CNN + biLSTMArchitecture=CNN and biLSTM hybrid2017.12 | 74.09 | 64.95 | 2.46 | 57.85 | 74.87 | 72.89 | — | |
| Stacked bi-LSTMs + shortcuts + max-pooling + attentionarchitecture=Stacked bi-LSTMs, shortcuts=true, pooling=max-pooling, attention=true2018.11 | 66.47 | — | 6.74 | 73.32 | 95.25 | — | 81.16 | |
| biLSTM + AttentionArchitecture=Bidirectional LSTM with Attention2017.12 | 58.74 | 63.17 | 0.03 | 63.48 | 77.49 | 73.27 | — | |
| SOLAT in the SWENCategory=Top-4 submission2017.12 | 58.5 | 82.05 | 1.86 | 76.18 | 98.7 | 89.08 | — | |
| Team SOLAT in the SWENrank=1st place at FNC-12018.11 | 58.5 | — | 1.86 | 76.18 | 98.7 | — | 82.02 | |
| Stacked bi-LSTMs + shortcuts + max-poolingarchitecture=Stacked bi-LSTMs, shortcuts=true, pooling=max-pooling2018.11 | 56.59 | — | 11.91 | 79.39 | 96.13 | — | 82.16 | |
| Chips Ahoy!Category=Top-4 submission2017.12 | 55.96 | 80.12 | 0.28 | 70.29 | 98.98 | 88.01 | — | |
| Stacked bi-GRUs + shortcuts + max-pooling + attentionarchitecture=Stacked bi-GRUs, shortcuts=true, pooling=max-pooling, attention=true2018.11 | 52.86 | — | 19.08 | 77.71 | 96.22 | — | 81.95 | |
| MLP with the considered external featuresarchitecture=MLP, features=external2018.11 | 52.55 | — | 2.87 | 77.93 | 97.86 | — | 81.95 | |
| Bi-GRU + max-pooling + attentionarchitecture=Bi-GRU, pooling=max-pooling, attention=true2018.11 | 52.08 | — | 4.16 | 76.08 | 97.36 | — | 80.76 | |
| Bi-LSTM + max-pooling + attention (best encoder)architecture=Bi-LSTM, pooling=max-pooling, attention=true, selection=best encoder2018.11 | 51.34 | — | 10.33 | 81.52 | 96.74 | — | 82.23 | |
| Word2vec + External FeaturesFeatures=Word2vec, External2017.12 | 50.7 | 75.78 | 9.61 | 53.38 | 96.05 | 82.79 | — | |
| Baseline based on word2vec + hand-crafted featuresfeatures=word2vec + hand-crafted features2018.11 | 50.7 | — | 9.61 | 53.38 | 96.05 | — | 72.78 | |
| Bi-LSTM + max-poolingarchitecture=Bi-LSTM, pooling=max-pooling2018.11 | 45.35 | — | 5.16 | 80.76 | 96.99 | — | 81.29 | |
| AtheneCategory=Top-4 submission2017.12 | 44.72 | 81.97 | 9.47 | 80.89 | 99.25 | 89.5 | — | |
| Team Athenerank=2nd place at FNC-12018.11 | 44.72 | — | 9.47 | 80.89 | 99.25 | — | 81.97 | |
| TF-IDF baselineType=Baseline2017.12 | 44.04 | 81.72 | 6.6 | 81.38 | 97.9 | 88.46 | — | |
| UCL Machine ReadingCategory=Top-4 submission2017.12 | 44.04 | 81.72 | 6.6 | 81.38 | 97.9 | 88.46 | — | |
| Baseline based on TF-IDF vectorsfeatures=TF-IDF vectors2018.11 | 44.04 | — | 6.6 | 81.38 | 97.9 | — | 81.72 | |
| Team UCL Machine Readingrank=3rd place at FNC-12018.11 | 44.04 | — | 6.6 | 81.38 | 97.9 | — | 81.72 | |
| ProposedFeatures=Neural, Statistical and External features2017.12 | 43.82 | 83.08 | 6.31 | 85.68 | 98.04 | 89.29 | — | |
| Previous state-of-the-art – Bhatt et al.2018.11 | 43.82 | — | 6.31 | 85.68 | 98.04 | — | 83.08 | |
| biLSTMArchitecture=Bidirectional LSTM2017.12 | 38.04 | 63.11 | 4.59 | 58.13 | 78.27 | 69.88 | — | |
| Neural baseline based on bi-directional LSTMsarchitecture=bi-directional LSTMs2018.11 | 38.04 | — | 4.59 | 58.13 | 78.27 | — | 63.11 | |
| CNNArchitecture=Convolutional Neural Network2017.12 | 35.89 | 60.91 | 2.1 | 46.77 | 88.47 | 74.84 | — | |
| Skip-thought baselineType=Baseline2017.12 | 31.8 | 76.18 | 0 | 81.2 | 91.18 | 82.48 | — | |
| Baseline based on skip-thought embeddingsfeatures=skip-thought embeddings2018.11 | 31.8 | — | 0 | 81.2 | 91.18 | — | 76.18 | |
| FNC-1 baselineType=Baseline2017.12 | 9.09 | 75.2 | 1 | 79.65 | 97.97 | 85.44 | — | |
| FNC-1 baseline2018.11 | 9.09 | — | 1 | 79.65 | 97.97 | — | 75.2 | |
| Best set of features from Tosik et al.2018.11 | 1.42 | — | 0 | 90.95 | 97.98 | — | 78.63 | |
| Neural method from Mohtarami et al.2018.11 | — | — | — | — | — | — | 78.97 | |
| Neural method from Mohtarami et al. + TF-IDFfeatures=TF-IDF2018.11 | — | — | — | — | — | — | 81.23 |