Natural Language Inference on HANS (test)
78.65AccuracyRoberta-large w/ Z-Aug
Evaluation results
| Method | Links | |
|---|---|---|
| Roberta-large w/ Z-AugTraining Data=DMNLI (Debiased MNLI), Backbone=Roberta-large, Debiasing Method=Z-Aug2022.03 | 78.65 | |
| Roberta-large w/ DMNLITraining Data=DMNLI (Debiased MNLI), Backbone=Roberta-large2022.03 | 75.74 | |
| MT-NLGshot_setting=Few-shot2022.01 | 73.16 | |
| E2E Self-debiasTraining Data=MNLI2022.03 | 71.2 | |
| SCILL-RExSelection Strategy=Oracle, Penalty=REx2022.06 | 71.2 | |
| SCILL-PGISelection Strategy=Oracle, Penalty=PGI2022.06 | 70.99 | |
| SCILL-cMMDSelection Strategy=TEV, Penalty=cMMD2022.06 | 70.92 | |
| SCILL-cMMDSelection Strategy=Oracle, Penalty=cMMD2022.06 | 70.77 | |
| Roberta-base w/ TAILORTraining Data=SNLI, Backbone=Roberta-base2022.03 | 70.5 | |
| SCILL-IRMSelection Strategy=TEV, Penalty=IRM2022.06 | 69.82 | |
| SCILL-RExSelection Strategy=TEV, Penalty=REx2022.06 | 69.75 | |
| SCILL-RExSelection Strategy=ID, Penalty=REx2022.06 | 69.73 | |
| DFLTraining Data=MNLI2022.03 | 69.26 | |
| PoESelection Strategy=ID, Penalty=-2022.06 | 69.2 | |
| SCILL-cMMDSelection Strategy=ID, Penalty=cMMD2022.06 | 69.15 | |
| SCILL-IRMSelection Strategy=ID, Penalty=IRM2022.06 | 69.11 | |
| Regularized-confTraining Data=MNLI2022.03 | 69.1 | |
| ConfRegSelection Strategy=ID, Penalty=-2022.06 | 69.1 | |
| BERT-base + PoE w/ Z-AugTraining Data=DMNLI (Debiased MNLI), Backbone=BERT-base, Debiasing Method=Z-Aug + PoE2022.03 | 68.75 | |
| SCILL-IRMSelection Strategy=Oracle, Penalty=IRM2022.06 | 68.72 | |
| SCILL-PGISelection Strategy=ID, Penalty=PGI2022.06 | 68.57 | |
| SCILL-PGISelection Strategy=TEV, Penalty=PGI2022.06 | 68.57 | |
| PoE+CETraining Data=MNLI, Backbone=BERT-base2022.03 | 67.9 | |
| BERT-base w/ Seq-ZTraining Data=DMNLI (Debiased MNLI), Backbone=BERT-base, Debiasing Method=Seq-Z2022.03 | 67.69 | |
| Roberta-base w/ Seq-ZTraining Data=DSNLI (Debiased SNLI), Backbone=Roberta-base, Debiasing Method=Seq-Z2022.03 | 66.87 | |
| EIIL-RExSelection Strategy=Oracle, Penalty=REx2022.06 | 66.87 | |
| EIIL-RExSelection Strategy=TEV, Penalty=REx2022.06 | 66.43 | |
| EIIL-IRMSelection Strategy=Oracle, Penalty=IRM2022.06 | 66.42 | |
| PoETraining Data=MNLI2022.03 | 66.31 | |
| Learned-Mixin+HTraining Data=MNLI2022.03 | 66.15 | |
| EIIL-PGISelection Strategy=Oracle, Penalty=PGI2022.06 | 66.02 | |
| EIIL-PGISelection Strategy=ID, Penalty=PGI2022.06 | 65.57 | |
| EIIL-PGISelection Strategy=TEV, Penalty=PGI2022.06 | 65.57 | |
| EIIL-IRMSelection Strategy=ID, Penalty=IRM2022.06 | 65.35 | |
| EIIL-IRMSelection Strategy=TEV, Penalty=IRM2022.06 | 65.35 | |
| Roberta-base w/ DSNLITraining Data=DSNLI (Debiased SNLI), Backbone=Roberta-base2022.03 | 65.32 | |
| EIIL-RExSelection Strategy=ID, Penalty=REx2022.06 | 65.16 | |
| BERT-base w/ Par-ZTraining Data=DMNLI (Debiased MNLI), Backbone=BERT-base, Debiasing Method=Par-Z2022.03 | 65.11 | |
| ERMSelection Strategy=ID, Penalty=-2022.06 | 64.88 | |
| ERMSelection Strategy=Oracle, Penalty=-2022.06 | 64.88 | |
| EIIL-cMMDSelection Strategy=Oracle, Penalty=cMMD2022.06 | 64.25 | |
| Learned-MixinTraining Data=MNLI2022.03 | 64 | |
| BERT-base + PoE w/ DMNLI (baseline)Training Data=DMNLI (Debiased MNLI), Backbone=BERT-base, Debiasing Method=PoE2022.03 | 63.4 | |
| EIIL-cMMDSelection Strategy=ID, Penalty=cMMD2022.06 | 63.22 | |
| EIIL-cMMDSelection Strategy=TEV, Penalty=cMMD2022.06 | 62.72 | |
| BERT-base w/ Z-AugTraining Data=DMNLI (Debiased MNLI), Backbone=BERT-base, Debiasing Method=Z-Aug2022.03 | 62.57 | |
| MT-NLGshot_setting=One-shot2022.01 | 60.01 | |
| Roberta-large w/ AFLiteTraining Data=SNLI, Backbone=Roberta-large2022.03 | 59.6 | |
| BERT-base AttentionTraining Data=SNLI, Backbone=BERT-base2022.03 | 58.42 | |
| GPT-2shot_setting=Zero-shot2022.01 | 54.79 | |
| BERT-base w/ DMNLI baselineTraining Data=DMNLI (Debiased MNLI), Backbone=BERT-base2022.03 | 54.36 | |
| MT-NLGshot_setting=Zero-shot2022.01 | 51.61 | |
| GPT-2shot_setting=One-shot2022.01 | 49.92 | |
| GPT-2shot_setting=Few-shot2022.01 | 49.79 |