Natural Language Inference on WNLI (test)
89AccuracyHNNensemble
Evaluation results
| Method | Links | |
|---|---|---|
| HNNensembleensemble=six models2019.07 | 89 | |
| HNN2019.07 | 83.6 | |
| BERT_CS_largeFormat=MCQA2019.08 | 83.6 | |
| BERT_largeFormat=MCQA2019.08 | 78.5 | |
| BERTWiki-WSCR2019.07 | 71.9 | |
| Kocijan et al. (2019)2019.08 | 71.9 | |
| BERTWSCR2019.07 | 70.5 | |
| BERT_largeTraining Objective=MTP2019.08 | 70.5 | |
| BERTLARGE-LM2019.07 | 65.1 | |
| FLAN-T5Model Variant=large, Parameters=780M, Evaluation Protocol=Zero-shot2023.07 | 62 | |
| FedAvgProportion of Attackers=0%, Backbone=DistilBERT, Attack Type=Gradient Manipulation2024.05 | 61.9 | |
| BREAProportion of Attackers=0%, Backbone=DistilBERT, Attack Type=Gradient Manipulation2024.05 | 61.9 | |
| RFLPAProportion of Attackers=0%, Backbone=DistilBERT, Attack Type=Gradient Manipulation2024.05 | 61.9 | |
| BREAProportion of Attackers=10%, Backbone=DistilBERT, Attack Type=Gradient Manipulation2024.05 | 59.2 | |
| RFLPAProportion of Attackers=10%, Backbone=DistilBERT, Attack Type=Gradient Manipulation2024.05 | 59.2 | |
| RFLPAProportion of Attackers=20%, Backbone=DistilBERT, Attack Type=Gradient Manipulation2024.05 | 59.2 | |
| BREAProportion of Attackers=20%, Backbone=DistilBERT, Attack Type=Gradient Manipulation2024.05 | 57.7 | |
| FedAvgProportion of Attackers=10%, Backbone=DistilBERT, Attack Type=Gradient Manipulation2024.05 | 56.3 | |
| BREAProportion of Attackers=30%, Backbone=DistilBERT, Attack Type=Gradient Manipulation2024.05 | 56.3 | |
| RFLPAProportion of Attackers=30%, Backbone=DistilBERT, Attack Type=Gradient Manipulation2024.05 | 56.3 | |
| ALIGNModel Variant=large, Parameters=355M, Evaluation Protocol=Zero-shot2023.07 | 54.9 | |
| ALIGNModel Variant=base, Parameters=125M, Evaluation Protocol=Zero-shot2023.07 | 52.1 | |
| FLAN-T5Model Variant=base, Parameters=250M, Evaluation Protocol=Zero-shot2023.07 | 46.5 | |
| FedAvgProportion of Attackers=20%, Backbone=DistilBERT, Attack Type=Gradient Manipulation2024.05 | 43.7 | |
| FedAvgProportion of Attackers=30%, Backbone=DistilBERT, Attack Type=Gradient Manipulation2024.05 | 43.7 |