Word Sense Disambiguation on WiC (dev)
89.5AccuracyROBERTa-WL+ w/ COSINE
Evaluation results
| Method | Links | |
|---|---|---|
| ROBERTa-WL+ w/ COSINELearning Setup=Transductive, Number of Parameters=125M, Access to weak labels=true2020.10 | 89.5 | |
| ROBERTa-WL+ w/ VATLearning Setup=Transductive, Number of Parameters=125M, Access to weak labels=true2020.10 | 84.9 | |
| ROBERTa-WL+ w/ MTLearning Setup=Transductive, Number of Parameters=125M, Access to weak labels=true2020.10 | 82.1 | |
| ROBERTa-WL+Learning Setup=Transductive, Number of Parameters=125M, Access to weak labels=true2020.10 | 81.3 | |
| SnorkelLearning Setup=Transductive, Number of Parameters=1M2020.10 | 80.5 | |
| Human BaselineLearning Setup=Human2020.10 | 80 | |
| ROBERTa-WL+ w/ COSINELearning Setup=Semi-Supervised, Number of Parameters=125M, Access to weak labels=true2020.10 | 76 | |
| ROBERTa-WL+ w/ VATLearning Setup=Semi-Supervised, Number of Parameters=125M, Access to weak labels=true2020.10 | 74.2 | |
| ROBERTa-WL+ w/ MTLearning Setup=Semi-Supervised, Number of Parameters=125M, Access to weak labels=true2020.10 | 73.5 | |
| ROBERTa-WL+Learning Setup=Semi-Supervised, Number of Parameters=125M, Access to weak labels=true2020.10 | 72.3 | |
| SenseBERTLearning Setup=Semi-Supervised, Number of Parameters=370M2020.10 | 72.1 | |
| ROBERTa-CLTraining supervision type=clean labels, Method categorization=Fully-supervised2020.10 | 70.53 | |
| ROBERTaLearning Setup=Standard, Number of Parameters=356M2020.10 | 70.5 | |
| COSINEMethod categorization=Framework2020.10 | 67.71 | |
| MixupMethod categorization=Baseline2020.10 | 64.88 | |
| SMARTMethod categorization=Baseline2020.10 | 63.55 | |
| USTMethod categorization=Baseline2020.10 | 63.48 | |
| InitMethod categorization=Framework2020.10 | 63.46 | |
| FreeLBMethod categorization=Baseline2020.10 | 63.45 | |
| Self-ensembleMethod categorization=Baseline2020.10 | 62.71 | |
| DenoiseMethod categorization=Baseline2020.10 | 62.38 | |
| ROBERTa-WL+Training supervision type=weak labels, Method categorization=Baseline2020.10 | 59.36 | |
| ExMatchMethod categorization=Baseline2020.10 | 58.8 | |
| Megatron-NLGModel Size=530B, Evaluation Protocol=Few-shot2021.12 | 58.5 | |
| GLaMModel Size=64B/64E, Evaluation Protocol=Few-shot, Shots=42021.12 | 56.3 | |
| GPT-3Model Size=175B, Evaluation Protocol=Few-shot, Shots=322021.12 | 55.3 | |
| ImplyLossMethod categorization=Baseline2020.10 | 54.48 | |
| GLaMModel Size=64B/64E, Evaluation Protocol=One-shot2021.12 | 52.7 | |
| GLaMModel Size=64B/64E, Evaluation Protocol=Zero-shot2021.12 | 50.3 | |
| GPT-3Model Size=175B, Evaluation Protocol=One-shot2021.12 | 48.6 | |
| WeSTClassMethod categorization=Baseline2020.10 | 48.59 | |
| GPT-3Model Size=175B, Evaluation Protocol=Zero-shot2021.12 | 0 |