Word Sense Disambiguation on WiC (test)
85.3AccuracyROBERTa-WL+ w/ COSINE
Evaluation results
| Method | Links | |
|---|---|---|
| ROBERTa-WL+ w/ COSINELearning Setup=Transductive, Number of Parameters=125M, Access to weak labels=true2020.10 | 85.3 | |
| ROBERTa-WL+ w/ VATLearning Setup=Transductive, Number of Parameters=125M, Access to weak labels=true2020.10 | 79.5 | |
| T5 + UDGsetting=supervised2022.01 | 77.9 | |
| ROBERTa-WL+ w/ MTLearning Setup=Transductive, Number of Parameters=125M, Access to weak labels=true2020.10 | 77.1 | |
| T5Learning Setup=Standard, Number of Parameters=11,000M2020.10 | 76.9 | |
| ROBERTa-WL+Learning Setup=Transductive, Number of Parameters=125M, Access to weak labels=true2020.10 | 76.8 | |
| SemEq-Large (+WSI)Parameters=355M2021.10 | 75.9 | |
| ROBERTa-WL+ w/ COSINELearning Setup=Semi-Supervised, Number of Parameters=125M, Access to weak labels=true2020.10 | 73.2 | |
| BERT_ARESParameters=342M2021.10 | 72.2 | |
| SenseBERT_LargeParameters=380M2021.10 | 72.1 | |
| T5-3BParameters=3000M2021.10 | 72.1 | |
| ROBERTa-WL+ w/ VATLearning Setup=Semi-Supervised, Number of Parameters=125M, Access to weak labels=true2020.10 | 71.2 | |
| ROBERTa-WL+ w/ MTLearning Setup=Semi-Supervised, Number of Parameters=125M, Access to weak labels=true2020.10 | 70.9 | |
| KnowBERT_W+WParameters=523M2021.10 | 70.9 | |
| ROBERTa-WL+Learning Setup=Semi-Supervised, Number of Parameters=125M, Access to weak labels=true2020.10 | 70.2 | |
| ROBERTaLearning Setup=Standard, Number of Parameters=356M2020.10 | 69.9 | |
| RoBERTa_LargeParameters=355M2021.10 | 69.9 | |
| BERTLearning Setup=Standard, Number of Parameters=335M2020.10 | 69.6 | |
| BERT_LargeParameters=340M2021.10 | 69.6 | |
| T5-LargeParameters=770M2021.10 | 69.3 | |
| ZoVHModel=OPT-1.3B, N=1, Forward pass budget=10,0002026.05 | 58.73 | |
| MT-NLGsetting=few-shot2022.01 | 58.46 | |
| MeZOModel=OPT-1.3B, Forward pass budget=10,0002026.05 | 57.26 | |
| HiZOOModel=OPT-1.3B, Forward pass budget=10,0002026.05 | 57.05 | |
| ZoVHModel=OPT-1.3B, N=2, Forward pass budget=10,0002026.05 | 56.64 | |
| ZoVHModel=OPT-13B, N=2, Forward pass budget=10,0002026.05 | 55.64 | |
| GPT-3setting=few-shot2022.01 | 55.3 | |
| HiZOOModel=OPT-13B, Forward pass budget=10,0002026.05 | 54.75 | |
| ZoVHModel=OPT-13B, N=1, Forward pass budget=10,0002026.05 | 54.34 | |
| MeZOModel=OPT-13B, Forward pass budget=10,0002026.05 | 54.08 | |
| MT-NLGsetting=one-shot2022.01 | 51.25 | |
| GPT-3setting=one-shot2022.01 | 48.6 | |
| MT-NLGsetting=zero-shot2022.01 | 48.59 | |
| GPT-3setting=zero-shot2022.01 | 0 |