Natural Language Inference on RTE (Multi-Run Metrics)
81.2Avg AccuracyT0-11B
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| T0-11BParams=11B2022.10 | 81.2 | — | 3.7 | |
| KiC-LargeParams=0.77B2022.10 | 74 | — | 3.8 | |
| KiC-BaseParams=0.22B2022.10 | 66.8 | — | 2.9 | |
| Logan IV et al. (2021)2022.04 | 64.4 | 58.5 | 3.9 | |
| T0-BaseParams=0.22B2022.10 | 64.1 | — | 1.4 | |
| T0-3BParams=3B2022.10 | 64.1 | — | 3.5 | |
| PETStrategy=Best2022.04 | 62.3 | 51.3 | 4.5 | |
| T0-LargeParams=0.77B2022.10 | 62.1 | — | 3.1 | |
| bitfit+mteMode=Ablation2022.04 | 61.3 | 53.8 | 5.2 | |
| PERFECTInitialization=prototype-based2022.04 | 60.7 | 52.7 | 4.5 | |
| PERFECTInitialization=random2022.04 | 60.4 | 53.1 | 4.7 | |
| PETStrategy=Average2022.04 | 60.1 | 49.5 | 4.7 | |
| prompt+mteMode=Ablation2022.04 | 56.9 | 50.7 | 4.1 | |
| FINETUNE2022.04 | 56.8 | 50.2 | 3.5 | |
| RoBERTaParams=0.35B2022.10 | 54.1 | — | 1.1 | |
| GPT-NeoParams=2.7B2022.10 | 51.1 | — | 3.2 | |
| GPT-JParams=6B2022.10 | 50.5 | — | 3.2 | |
| BERTParams=0.34B2022.10 | 48.4 | — | 1.7 | |
| GPT-NeoXParams=20B2022.10 | 48.4 | — | 2.7 | |
| OPTParams=30B2022.10 | 47.3 | — | 1.7 | |
| KiC-SmallParams=0.06B2022.10 | 47.3 | — | 2.4 |