# Word Sense Disambiguation on WiC (test) Source: [Word Sense Disambiguation on WiC (test)]() Collection index: [llms.txt]() Data updated: 2026-06-01T08:37:56.081Z Evaluation of Word Sense Disambiguation performance on the test set of the WiC dataset. A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. | Scope | Recorded value | | --- | --- | | Task | Word Sense Disambiguation | | Dataset (including any stated split) | WiC (test) | | Metric | Accuracy | | Unit | PERCENT | | Direction | Higher is better | | Results loaded | 34 of 34 | | Coverage | All results currently returned for this benchmark | Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol. ## Metrics - [Accuracy](): Higher is better; unit: PERCENT. ## Recorded results | Method | Accuracy | Result date | Report paper | | --- | --- | --- | --- | | ROBERTa-WL+ w/ COSINE | 85.3 | Not recorded | 68fa699a1e209131cea23031 | | ROBERTa-WL+ w/ VAT | 79.5 | Not recorded | 68fa699a1e209131cea23031 | | T5 + UDG | 77.9 | Not recorded | 68fa6bb61e209131cea2fcb1 | | ROBERTa-WL+ w/ MT | 77.1 | Not recorded | 68fa699a1e209131cea23031 | | T5 | 76.9 | Not recorded | 68fa699a1e209131cea23031 | | ROBERTa-WL+ | 76.8 | Not recorded | 68fa699a1e209131cea23031 | | SemEq-Large (+WSI) | 75.9 | Not recorded | 68fa6b171e209131cea2c152 | | ROBERTa-WL+ w/ COSINE | 73.2 | Not recorded | 68fa699a1e209131cea23031 | | BERT\_ARES | 72.2 | Not recorded | 68fa6b171e209131cea2c152 | | SenseBERT\_Large | 72.1 | Not recorded | 68fa6b171e209131cea2c152 | | T5-3B | 72.1 | Not recorded | 68fa6b171e209131cea2c152 | | ROBERTa-WL+ w/ VAT | 71.2 | Not recorded | 68fa699a1e209131cea23031 | | ROBERTa-WL+ w/ MT | 70.9 | Not recorded | 68fa699a1e209131cea23031 | | KnowBERT\_W+W | 70.9 | Not recorded | 68fa6b171e209131cea2c152 | | ROBERTa-WL+ | 70.2 | Not recorded | 68fa699a1e209131cea23031 | | ROBERTa | 69.9 | Not recorded | 68fa699a1e209131cea23031 | | RoBERTa\_Large | 69.9 | Not recorded | 68fa6b171e209131cea2c152 | | BERT | 69.6 | Not recorded | 68fa699a1e209131cea23031 | | BERT\_Large | 69.6 | Not recorded | 68fa6b171e209131cea2c152 | | T5-Large | 69.3 | Not recorded | 68fa6b171e209131cea2c152 | | ZoVH | 58.73 | Not recorded | 6a1d11e5dbd7150506f83e0e | | MT-NLG | 58.46 | Not recorded | 68fa6bb61e209131cea2fcb1 | | MeZO | 57.26 | Not recorded | 6a1d11e5dbd7150506f83e0e | | HiZOO | 57.05 | Not recorded | 6a1d11e5dbd7150506f83e0e | | ZoVH | 56.64 | Not recorded | 6a1d11e5dbd7150506f83e0e | | ZoVH | 55.64 | Not recorded | 6a1d11e5dbd7150506f83e0e | | GPT-3 | 55.3 | Not recorded | 68fa6bb61e209131cea2fcb1 | | HiZOO | 54.75 | Not recorded | 6a1d11e5dbd7150506f83e0e | | ZoVH | 54.34 | Not recorded | 6a1d11e5dbd7150506f83e0e | | MeZO | 54.08 | Not recorded | 6a1d11e5dbd7150506f83e0e | | MT-NLG | 51.25 | Not recorded | 68fa6bb61e209131cea2fcb1 | | GPT-3 | 48.6 | Not recorded | 68fa6bb61e209131cea2fcb1 | | MT-NLG | 48.59 | Not recorded | 68fa6bb61e209131cea2fcb1 | | GPT-3 | 0 | Not recorded | 68fa6bb61e209131cea2fcb1 | ### ROBERTa-WL+ w/ COSINE Accuracy: 85.3. Result ID: 698dc2e78b517eb4332390cf. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Learning Setup":"Transductive","Number of Parameters":"125M","Access to weak labels":true}. This record does not assert independent reproduction. ### ROBERTa-WL+ w/ VAT Accuracy: 79.5. Result ID: 698dc2e78b517eb4332390ce. [Reporting paper]() Recorded evaluation configuration: {"Learning Setup":"Transductive","Number of Parameters":"125M","Access to weak labels":true}. This record does not assert independent reproduction. ### T5 + UDG Accuracy: 77.9. Result ID: 698e9f8affc3901de09f3b4b. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"setting":"supervised"}. This record does not assert independent reproduction. ### ROBERTa-WL+ w/ MT Accuracy: 77.1. Result ID: 698dc2e78b517eb4332390cd. [Reporting paper]() Recorded evaluation configuration: {"Learning Setup":"Transductive","Number of Parameters":"125M","Access to weak labels":true}. This record does not assert independent reproduction. ### T5 Accuracy: 76.9. Result ID: 698dc2e78b517eb4332390c7. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Learning Setup":"Standard","Number of Parameters":"11,000M"}. This record does not assert independent reproduction. ### ROBERTa-WL+ Accuracy: 76.8. Result ID: 698dc2e78b517eb4332390cc. [Reporting paper]() Recorded evaluation configuration: {"Learning Setup":"Transductive","Number of Parameters":"125M","Access to weak labels":true}. This record does not assert independent reproduction. ### SemEq-Large (+WSI) Accuracy: 75.9. Result ID: 698e5847ffc3901de09ccfdc. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Parameters":"355M"}. This record does not assert independent reproduction. ### ROBERTa-WL+ w/ COSINE Accuracy: 73.2. Result ID: 698dc2e78b517eb4332390cb. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Learning Setup":"Semi-Supervised","Number of Parameters":"125M","Access to weak labels":true}. This record does not assert independent reproduction. ### BERT\_ARES Accuracy: 72.2. Result ID: 698e5847ffc3901de09ccfdb. [Reporting paper]() Recorded evaluation configuration: {"Parameters":"342M"}. This record does not assert independent reproduction. ### SenseBERT\_Large Accuracy: 72.1. Result ID: 698e5847ffc3901de09ccfd8. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Parameters":"380M"}. This record does not assert independent reproduction. ### T5-3B Accuracy: 72.1. Result ID: 698e5847ffc3901de09ccfda. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Parameters":"3000M"}. This record does not assert independent reproduction. ### ROBERTa-WL+ w/ VAT Accuracy: 71.2. Result ID: 698dc2e78b517eb4332390ca. [Reporting paper]() Recorded evaluation configuration: {"Learning Setup":"Semi-Supervised","Number of Parameters":"125M","Access to weak labels":true}. This record does not assert independent reproduction. ### ROBERTa-WL+ w/ MT Accuracy: 70.9. Result ID: 698dc2e78b517eb4332390c9. [Reporting paper]() Recorded evaluation configuration: {"Learning Setup":"Semi-Supervised","Number of Parameters":"125M","Access to weak labels":true}. This record does not assert independent reproduction. ### KnowBERT\_W+W Accuracy: 70.9. Result ID: 698e5847ffc3901de09ccfd7. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Parameters":"523M"}. This record does not assert independent reproduction. ### ROBERTa-WL+ Accuracy: 70.2. Result ID: 698dc2e78b517eb4332390c8. [Reporting paper]() Recorded evaluation configuration: {"Learning Setup":"Semi-Supervised","Number of Parameters":"125M","Access to weak labels":true}. This record does not assert independent reproduction. ### ROBERTa Accuracy: 69.9. Result ID: 698dc2e78b517eb4332390c6. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Learning Setup":"Standard","Number of Parameters":"356M"}. This record does not assert independent reproduction. ### RoBERTa\_Large Accuracy: 69.9. Result ID: 698e5847ffc3901de09ccfd6. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Parameters":"355M"}. This record does not assert independent reproduction. ### BERT Accuracy: 69.6. Result ID: 698dc2e78b517eb4332390c5. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Learning Setup":"Standard","Number of Parameters":"335M"}. This record does not assert independent reproduction. ### BERT\_Large Accuracy: 69.6. Result ID: 698e5847ffc3901de09ccfd5. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Parameters":"340M"}. This record does not assert independent reproduction. ### T5-Large Accuracy: 69.3. Result ID: 698e5847ffc3901de09ccfd9. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Parameters":"770M"}. This record does not assert independent reproduction. ### ZoVH Accuracy: 58.73. Result ID: 6a1d44d1fbc373592c167e58. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"OPT-1.3B","N":"1","Forward pass budget":"10,000"}. This record does not assert independent reproduction. ### MT-NLG Accuracy: 58.46. Result ID: 698e9f8affc3901de09f3b4a. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"setting":"few-shot"}. This record does not assert independent reproduction. ### MeZO Accuracy: 57.26. Result ID: 6a1d44d1fbc373592c167e56. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"OPT-1.3B","Forward pass budget":"10,000"}. This record does not assert independent reproduction. ### HiZOO Accuracy: 57.05. Result ID: 6a1d44d1fbc373592c167e57. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"OPT-1.3B","Forward pass budget":"10,000"}. This record does not assert independent reproduction. ### ZoVH Accuracy: 56.64. Result ID: 6a1d44d1fbc373592c167e59. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"OPT-1.3B","N":"2","Forward pass budget":"10,000"}. This record does not assert independent reproduction. ### ZoVH Accuracy: 55.64. Result ID: 6a1d44d1fbc373592c167e5d. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"OPT-13B","N":"2","Forward pass budget":"10,000"}. This record does not assert independent reproduction. ### GPT-3 Accuracy: 55.3. Result ID: 698e9f8affc3901de09f3b47. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"setting":"few-shot"}. This record does not assert independent reproduction. ### HiZOO Accuracy: 54.75. Result ID: 6a1d44d1fbc373592c167e5b. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"OPT-13B","Forward pass budget":"10,000"}. This record does not assert independent reproduction. ### ZoVH Accuracy: 54.34. Result ID: 6a1d44d1fbc373592c167e5c. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"OPT-13B","N":"1","Forward pass budget":"10,000"}. This record does not assert independent reproduction. ### MeZO Accuracy: 54.08. Result ID: 6a1d44d1fbc373592c167e5a. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"OPT-13B","Forward pass budget":"10,000"}. This record does not assert independent reproduction. ### MT-NLG Accuracy: 51.25. Result ID: 698e9f8affc3901de09f3b49. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"setting":"one-shot"}. This record does not assert independent reproduction. ### GPT-3 Accuracy: 48.6. Result ID: 698e9f8affc3901de09f3b46. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"setting":"one-shot"}. This record does not assert independent reproduction. ### MT-NLG Accuracy: 48.59. Result ID: 698e9f8affc3901de09f3b48. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"setting":"zero-shot"}. This record does not assert independent reproduction. ### GPT-3 Accuracy: 0. Result ID: 698e9f8affc3901de09f3b45. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"setting":"zero-shot"}. This record does not assert independent reproduction. Page 1 of 1. This is an index page, not the full collection.