# Word Sense Disambiguation on WiC (dev) Source: [Word Sense Disambiguation on WiC (dev)]() Collection index: [llms.txt]() Data updated: 2026-02-26T05:28:27.188Z Evaluation of Word Sense Disambiguation performance on the development set of the WiC dataset. A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. | Scope | Recorded value | | --- | --- | | Task | Word Sense Disambiguation | | Dataset (including any stated split) | WiC (dev) | | Metric | Accuracy | | Unit | PERCENT | | Direction | Higher is better | | Results loaded | 32 of 32 | | Coverage | All results currently returned for this benchmark | Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol. ## Metrics - [Accuracy](): Higher is better; unit: PERCENT. ## Recorded results | Method | Accuracy | Result date | Report paper | | --- | --- | --- | --- | | ROBERTa-WL+ w/ COSINE | 89.5 | Not recorded | 68fa699a1e209131cea23031 | | ROBERTa-WL+ w/ VAT | 84.9 | Not recorded | 68fa699a1e209131cea23031 | | ROBERTa-WL+ w/ MT | 82.1 | Not recorded | 68fa699a1e209131cea23031 | | ROBERTa-WL+ | 81.3 | Not recorded | 68fa699a1e209131cea23031 | | Snorkel | 80.5 | Not recorded | 68fa699a1e209131cea23031 | | Human Baseline | 80 | Not recorded | 68fa699a1e209131cea23031 | | ROBERTa-WL+ w/ COSINE | 76 | Not recorded | 68fa699a1e209131cea23031 | | ROBERTa-WL+ w/ VAT | 74.2 | Not recorded | 68fa699a1e209131cea23031 | | ROBERTa-WL+ w/ MT | 73.5 | Not recorded | 68fa699a1e209131cea23031 | | ROBERTa-WL+ | 72.3 | Not recorded | 68fa699a1e209131cea23031 | | SenseBERT | 72.1 | Not recorded | 68fa699a1e209131cea23031 | | ROBERTa-CL | 70.53 | Not recorded | 68fa699a1e209131cea23031 | | ROBERTa | 70.5 | Not recorded | 68fa699a1e209131cea23031 | | COSINE | 67.71 | Not recorded | 68fa699a1e209131cea23031 | | Mixup | 64.88 | Not recorded | 68fa699a1e209131cea23031 | | SMART | 63.55 | Not recorded | 68fa699a1e209131cea23031 | | UST | 63.48 | Not recorded | 68fa699a1e209131cea23031 | | Init | 63.46 | Not recorded | 68fa699a1e209131cea23031 | | FreeLB | 63.45 | Not recorded | 68fa699a1e209131cea23031 | | Self-ensemble | 62.71 | Not recorded | 68fa699a1e209131cea23031 | | Denoise | 62.38 | Not recorded | 68fa699a1e209131cea23031 | | ROBERTa-WL+ | 59.36 | Not recorded | 68fa699a1e209131cea23031 | | ExMatch | 58.8 | Not recorded | 68fa699a1e209131cea23031 | | Megatron-NLG | 58.5 | Not recorded | 68fa6d1d1e209131cea37ddb | | GLaM | 56.3 | Not recorded | 68fa6d1d1e209131cea37ddb | | GPT-3 | 55.3 | Not recorded | 68fa6d1d1e209131cea37ddb | | ImplyLoss | 54.48 | Not recorded | 68fa699a1e209131cea23031 | | GLaM | 52.7 | Not recorded | 68fa6d1d1e209131cea37ddb | | GLaM | 50.3 | Not recorded | 68fa6d1d1e209131cea37ddb | | GPT-3 | 48.6 | Not recorded | 68fa6d1d1e209131cea37ddb | | WeSTClass | 48.59 | Not recorded | 68fa699a1e209131cea23031 | | GPT-3 | 0 | Not recorded | 68fa6d1d1e209131cea37ddb | ### ROBERTa-WL+ w/ COSINE Accuracy: 89.5. Result ID: 698dc2e78b517eb4332390c4. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Learning Setup":"Transductive","Number of Parameters":"125M","Access to weak labels":true}. This record does not assert independent reproduction. ### ROBERTa-WL+ w/ VAT Accuracy: 84.9. Result ID: 698dc2e78b517eb4332390c3. [Reporting paper]() Recorded evaluation configuration: {"Learning Setup":"Transductive","Number of Parameters":"125M","Access to weak labels":true}. This record does not assert independent reproduction. ### ROBERTa-WL+ w/ MT Accuracy: 82.1. Result ID: 698dc2e78b517eb4332390c2. [Reporting paper]() Recorded evaluation configuration: {"Learning Setup":"Transductive","Number of Parameters":"125M","Access to weak labels":true}. This record does not assert independent reproduction. ### ROBERTa-WL+ Accuracy: 81.3. Result ID: 698dc2e78b517eb4332390c1. [Reporting paper]() Recorded evaluation configuration: {"Learning Setup":"Transductive","Number of Parameters":"125M","Access to weak labels":true}. This record does not assert independent reproduction. ### Snorkel Accuracy: 80.5. Result ID: 698dc2e78b517eb4332390c0. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Learning Setup":"Transductive","Number of Parameters":"1M"}. This record does not assert independent reproduction. ### Human Baseline Accuracy: 80. Result ID: 698dc2e78b517eb4332390b9. [Reporting paper]() Recorded evaluation configuration: {"Learning Setup":"Human"}. This record does not assert independent reproduction. ### ROBERTa-WL+ w/ COSINE Accuracy: 76. Result ID: 698dc2e78b517eb4332390bf. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Learning Setup":"Semi-Supervised","Number of Parameters":"125M","Access to weak labels":true}. This record does not assert independent reproduction. ### ROBERTa-WL+ w/ VAT Accuracy: 74.2. Result ID: 698dc2e78b517eb4332390be. [Reporting paper]() Recorded evaluation configuration: {"Learning Setup":"Semi-Supervised","Number of Parameters":"125M","Access to weak labels":true}. This record does not assert independent reproduction. ### ROBERTa-WL+ w/ MT Accuracy: 73.5. Result ID: 698dc2e78b517eb4332390bd. [Reporting paper]() Recorded evaluation configuration: {"Learning Setup":"Semi-Supervised","Number of Parameters":"125M","Access to weak labels":true}. This record does not assert independent reproduction. ### ROBERTa-WL+ Accuracy: 72.3. Result ID: 698dc2e78b517eb4332390bc. [Reporting paper]() Recorded evaluation configuration: {"Learning Setup":"Semi-Supervised","Number of Parameters":"125M","Access to weak labels":true}. This record does not assert independent reproduction. ### SenseBERT Accuracy: 72.1. Result ID: 698dc2e78b517eb4332390bb. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Learning Setup":"Semi-Supervised","Number of Parameters":"370M"}. This record does not assert independent reproduction. ### ROBERTa-CL Accuracy: 70.53. Result ID: 698dc2e78b517eb433239123. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Training supervision type":"clean labels","Method categorization":"Fully-supervised"}. This record does not assert independent reproduction. ### ROBERTa Accuracy: 70.5. Result ID: 698dc2e78b517eb4332390ba. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Learning Setup":"Standard","Number of Parameters":"356M"}. This record does not assert independent reproduction. ### COSINE Accuracy: 67.71. Result ID: 698dc2e78b517eb43323912e. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Method categorization":"Framework"}. This record does not assert independent reproduction. ### Mixup Accuracy: 64.88. Result ID: 698dc2e78b517eb433239127. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Method categorization":"Baseline"}. This record does not assert independent reproduction. ### SMART Accuracy: 63.55. Result ID: 698dc2e78b517eb433239128. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Method categorization":"Baseline"}. This record does not assert independent reproduction. ### UST Accuracy: 63.48. Result ID: 698dc2e78b517eb43323912c. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Method categorization":"Baseline"}. This record does not assert independent reproduction. ### Init Accuracy: 63.46. Result ID: 698dc2e78b517eb43323912d. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Method categorization":"Framework"}. This record does not assert independent reproduction. ### FreeLB Accuracy: 63.45. Result ID: 698dc2e78b517eb433239126. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Method categorization":"Baseline"}. This record does not assert independent reproduction. ### Self-ensemble Accuracy: 62.71. Result ID: 698dc2e78b517eb433239125. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Method categorization":"Baseline"}. This record does not assert independent reproduction. ### Denoise Accuracy: 62.38. Result ID: 698dc2e78b517eb43323912b. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Method categorization":"Baseline"}. This record does not assert independent reproduction. ### ROBERTa-WL+ Accuracy: 59.36. Result ID: 698dc2e78b517eb433239124. [Reporting paper]() Recorded evaluation configuration: {"Training supervision type":"weak labels","Method categorization":"Baseline"}. This record does not assert independent reproduction. ### ExMatch Accuracy: 58.8. Result ID: 698dc2e78b517eb433239122. [Reporting paper]() Recorded evaluation configuration: {"Method categorization":"Baseline"}. This record does not assert independent reproduction. ### Megatron-NLG Accuracy: 58.5. Result ID: 698f92f23a6c314849fa37fe. [Reporting paper]() Recorded evaluation configuration: {"Model Size":"530B","Evaluation Protocol":"Few-shot"}. This record does not assert independent reproduction. ### GLaM Accuracy: 56.3. Result ID: 698f92f23a6c314849fa37ff. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model Size":"64B/64E","Evaluation Protocol":"Few-shot","Shots":4}. This record does not assert independent reproduction. ### GPT-3 Accuracy: 55.3. Result ID: 698f92f23a6c314849fa37fd. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Model Size":"175B","Evaluation Protocol":"Few-shot","Shots":32}. This record does not assert independent reproduction. ### ImplyLoss Accuracy: 54.48. Result ID: 698dc2e78b517eb43323912a. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Method categorization":"Baseline"}. This record does not assert independent reproduction. ### GLaM Accuracy: 52.7. Result ID: 698f92f23a6c314849fa37fc. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model Size":"64B/64E","Evaluation Protocol":"One-shot"}. This record does not assert independent reproduction. ### GLaM Accuracy: 50.3. Result ID: 698f92f23a6c314849fa37fa. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model Size":"64B/64E","Evaluation Protocol":"Zero-shot"}. This record does not assert independent reproduction. ### GPT-3 Accuracy: 48.6. Result ID: 698f92f23a6c314849fa37fb. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Model Size":"175B","Evaluation Protocol":"One-shot"}. This record does not assert independent reproduction. ### WeSTClass Accuracy: 48.59. Result ID: 698dc2e78b517eb433239129. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Method categorization":"Baseline"}. This record does not assert independent reproduction. ### GPT-3 Accuracy: 0. Result ID: 698f92f23a6c314849fa37f9. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Model Size":"175B","Evaluation Protocol":"Zero-shot"}. This record does not assert independent reproduction. Page 1 of 1. This is an index page, not the full collection.