Fact Verification on FEVER (dev)
82.1Label AccuracyROBERTaBase Flexible (IE1) + MSPP (REk)
Evaluation results
| Method | Links | ||
|---|---|---|---|
| ROBERTaBase Flexible (IE1) + MSPP (REk)Backbone=RoBERTa-Base, Sequence Type=Flexible, Prediction Head=IE1, MSPP Pre-training=REk2023.09 | 82.1 | — | |
| Joint Pre IEk + FT IE1Backbone=RoBERTa-Base, Model Architecture=Joint, Pre-training Head=IEk, Fine-tuning Head=IE12022.05 | 81.21 | — | |
| Joint Pre AEk + FT AE1Backbone=RoBERTa-Base, Model Architecture=Joint, Pre-training Head=AEk, Fine-tuning Head=AE12022.05 | 81.21 | — | |
| Joint Pre IEk + FT IE1Pre-training head=IEk, Fine-tuning head=IE12022.05 | 81.21 | — | |
| Joint Pre AEk + FT AE1Pre-training head=AEk, Fine-tuning head=AE12022.05 | 81.21 | — | |
| ROBERTaBase Fixed (IE1) + MSPP (IEk)Backbone=RoBERTa-Base, Sequence Type=Fixed, Prediction Head=IE1, MSPP Pre-training=IEk2023.09 | 81.21 | — | |
| ROBERTaBase Fixed (AE1) + MSPP (AEk)Backbone=RoBERTa-Base, Sequence Type=Fixed, Prediction Head=AE1, MSPP Pre-training=AEk2023.09 | 81.21 | — | |
| Joint Pre AEk + FT IE1Backbone=RoBERTa-Base, Model Architecture=Joint, Pre-training Head=AEk, Fine-tuning Head=IE12022.05 | 81.18 | — | |
| Joint Pre AEk + FT IE1Pre-training head=AEk, Fine-tuning head=IE12022.05 | 81.18 | — | |
| Joint Pre IEk + FT AE1Backbone=RoBERTa-Base, Model Architecture=Joint, Pre-training Head=IEk, Fine-tuning Head=AE12022.05 | 81.1 | — | |
| Joint Pre IEk + FT AE1Pre-training head=IEk, Fine-tuning head=AE12022.05 | 81.1 | — | |
| ProoFVer-SBRetriever=Stammbach (2021)2021.08 | 80.74 | 79.07 | |
| DominikSRetriever=Stammbach (2021)2021.08 | 80.59 | 78.37 | |
| ProoFVerRetriever=Liu et al. (2020)2021.08 | 80.23 | 78.17 | |
| Pairwise RoBERTa-Base + WgtSumBackbone=RoBERTa-Base, Model Architecture=Pairwise, Aggregation=WgtSum2022.05 | 80.01 | — | |
| Pairwise RoBERTa-Base + WgtSumApproach=Pairwise, Backbone=RoBERTa-Base, Aggregation=WgtSum2022.05 | 80.01 | — | |
| ROBERTaBase Pairwise + WgtSumBackbone=RoBERTa-Base, Aggregation=WgtSum2023.09 | 80.01 | — | |
| ProoFVer-ARetriever=Liu et al. (2020)2021.08 | 79.83 | 76.33 | |
| Pairwise RoBERTa-Base + MaxPoolBackbone=RoBERTa-Base, Model Architecture=Pairwise, Aggregation=MaxPool2022.05 | 79.82 | — | |
| Pairwise RoBERTa-Base + MaxPoolApproach=Pairwise, Backbone=RoBERTa-Base, Aggregation=MaxPool2022.05 | 79.82 | — | |
| ROBERTaBase Pairwise + MaxPoolBackbone=RoBERTa-Base, Aggregation=MaxPool2023.09 | 79.82 | — | |
| Joint RoBERTa-BaseBackbone=RoBERTa-Base, Model Architecture=Joint, Fine-tuning Head=IE12022.05 | 79.25 | — | |
| Joint RoBERTa-Base + FT IE1Approach=Joint, Backbone=RoBERTa-Base, Fine-tuning head=IE12022.05 | 79.25 | — | |
| ROBERTaBase Flexible (IE1)Backbone=RoBERTa-Base, Sequence Type=Flexible, Prediction Head=IE1, MSPP Pre-training=false2023.09 | 79.25 | — | |
| DREAMBackbone=XL-Net, Retrieval Type=sophisticated2022.05 | 79.16 | — | |
| DREAMBackbone=XLNet-Large2023.09 | 79.16 | — | |
| CorefBERTRetriever=Liu et al. (2020)2021.08 | 79.12 | 77.46 | |
| ROBERTaBase Fixed (IE1)Backbone=RoBERTa-Base, Sequence Type=Fixed, Prediction Head=IE1, MSPP Pre-training=false2023.09 | 78.92 | — | |
| ROBERTaBase Fixed (AE1)Backbone=RoBERTa-Base, Sequence Type=Fixed, Prediction Head=AE1, MSPP Pre-training=false2023.09 | 78.85 | — | |
| ProoFVer-MVRetriever=Liu et al. (2020)2021.08 | 78.71 | 74.62 | |
| KGATBackbone=RoBERTa-Base2022.05 | 78.29 | — | |
| KGATBackbone=RoBERTa-Base2022.05 | 78.29 | — | |
| KGATRetriever=Liu et al. (2020)2021.08 | 78.29 | 76.11 | |
| KGAT with RoBERTaBaseBackbone=RoBERTa-Base2023.09 | 78.29 | — | |
| Transformer-XH2022.05 | 78.05 | — | |
| Transformer-XH2022.05 | 78.05 | — | |
| Transformer-XH2023.09 | 78.05 | — | |
| DOMLIN++Retrieval Type=sophisticated2022.05 | 77.48 | — | |
| DOMLIN++Backbone=RoBERTa-Large2023.09 | 77.48 | — | |
| ProoFVer-ARRetriever=Liu et al. (2020)2021.08 | 77.42 | 75.27 | |
| GEAR2019.07 | 74.84 | 70.69 | |
| RAG#Params=620M, #Encoded=1002021.10 | 74.5 | — | |
| BERT Concat2019.07 | 73.67 | 68.89 | |
| Joint BERT-BaseBackbone=BERT-Base, Model Architecture=Joint2022.05 | 73.67 | — | |
| BERT Pair2019.07 | 73.3 | 68.9 | |
| Pairwise BERT-BaseBackbone=BERT-Base, Model Architecture=Pairwise2022.05 | 73.3 | — | |
| DOMLIN2023.09 | 71.54 | — | |
| TOME-2#Params=220M, #Encoded=12021.10 | 71.1 | — | |
| GEAR2022.05 | 70.69 | — | |
| GEAR2022.05 | 70.69 | — | |
| GEAR2023.09 | 70.69 | — | |
| TOME-1#Params=220M, #Encoded=12021.10 | 70.5 | — | |
| REALM#Params=330M, #Encoded=52021.10 | 70.4 | — | |
| UNC NLP2019.07 | 69.72 | 66.49 | |
| UCL MRG2019.07 | 69.66 | 65.41 | |
| Athene2019.07 | 68.49 | 64.74 | |
| EaE#Params=360M, #Encoded=12021.10 | 66.1 | — |