# Natural Language Understanding on SuperGLUE (WSC, WIC, RTE, CB, MultiRC, BoolQ, COPA, AVG) Source: [Natural Language Understanding on SuperGLUE (WSC, WIC, RTE, CB, MultiRC, BoolQ, COPA, AVG)]() Collection index: [llms.txt]() Data updated: 2026-05-15T07:33:58.928Z Evaluation of Natural Language Understanding capabilities on the SuperGLUE benchmark, specifically reporting results for the tasks WSC, WIC, RTE, CB, MultiRC, BoolQ, COPA, and the overall average score. A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. | Scope | Recorded value | | --- | --- | | Task | Natural Language Understanding | | Dataset (including any stated split) | SuperGLUE | | Metric | WSC Score | | Unit | PERCENT | | Direction | Higher is better | | Results loaded | 25 of 25 | | Coverage | All results currently returned for this benchmark | Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol. ## Metrics - [WSC Score](): Higher is better; unit: PERCENT. - [WIC Score](): Higher is better; unit: PERCENT. - [RTE Score](): Higher is better; unit: PERCENT. - [CB Score](): Higher is better; unit: PERCENT. - [MultiRC Score](): Higher is better; unit: SCALAR. - [BoolQ Score](): Higher is better; unit: PERCENT. - [COPA Score](): Higher is better; unit: PERCENT. - [Average Score (AVG)](): Higher is better; unit: PERCENT. ## Recorded results | Method | WSC Score | Result date | Report paper | | --- | --- | --- | --- | | Poly | 76.9 | Not recorded | 6a06a899dbc836b87d82a056 | | MHR | 75.9 | Not recorded | 6a06a899dbc836b87d82a056 | | C-Poly | 75 | Not recorded | 6a06a899dbc836b87d82a056 | | PEML | 75 | Not recorded | 6a06a899dbc836b87d82a056 | | MOE-LoRA | 66.3 | Not recorded | 6a06a899dbc836b87d82a056 | | LoRA | 64.4 | Not recorded | 6a06a899dbc836b87d82a056 | | Pythia | 57.69 | Not recorded | 68fa711e1e209131cea4f3d4 | | PEML | 55.2 | Not recorded | 6a06a899dbc836b87d82a056 | | C-Poly | 53.4 | Not recorded | 6a06a899dbc836b87d82a056 | | Poly | 47 | Not recorded | 6a06a899dbc836b87d82a056 | | MHR | 45.5 | Not recorded | 6a06a899dbc836b87d82a056 | | BLOOM | 40.38 | Not recorded | 68fa711e1e209131cea4f3d4 | | HGRN | 40.38 | Not recorded | 68fa711e1e209131cea4f3d4 | | MOE-LoRA | 39.6 | Not recorded | 6a06a899dbc836b87d82a056 | | HGRN | 38.46 | Not recorded | 68fa711e1e209131cea4f3d4 | | HGRN | 38.46 | Not recorded | 68fa711e1e209131cea4f3d4 | | OPT | 37.5 | Not recorded | 68fa711e1e209131cea4f3d4 | | GPT-Neo | 36.54 | Not recorded | 68fa711e1e209131cea4f3d4 | | OPT | 36.54 | Not recorded | 68fa711e1e209131cea4f3d4 | | Pythia | 36.54 | Not recorded | 68fa711e1e209131cea4f3d4 | | OPT | 36.54 | Not recorded | 68fa711e1e209131cea4f3d4 | | GPT-Neo | 36.54 | Not recorded | 68fa711e1e209131cea4f3d4 | | Pythia | 36.54 | Not recorded | 68fa711e1e209131cea4f3d4 | | BLOOM | 36.54 | Not recorded | 68fa711e1e209131cea4f3d4 | | LoRA | 32.3 | Not recorded | 6a06a899dbc836b87d82a056 | ### Poly WSC Score: 76.9. Result ID: 6a06cc5e47f34ce924ee0f04. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"FLAN-T5-Large"}. This record does not assert independent reproduction. ### MHR WSC Score: 75.9. Result ID: 6a06cc5e47f34ce924ee0f05. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"FLAN-T5-Large"}. This record does not assert independent reproduction. ### C-Poly WSC Score: 75. Result ID: 6a06cc5e47f34ce924ee0f06. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"FLAN-T5-Large"}. This record does not assert independent reproduction. ### PEML WSC Score: 75. Result ID: 6a06cc5e47f34ce924ee0f07. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"FLAN-T5-Large"}. This record does not assert independent reproduction. ### MOE-LoRA WSC Score: 66.3. Result ID: 6a06cc5e47f34ce924ee0f03. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"FLAN-T5-Large"}. This record does not assert independent reproduction. ### LoRA WSC Score: 64.4. Result ID: 6a06cc5e47f34ce924ee0f02. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"FLAN-T5-Large"}. This record does not assert independent reproduction. ### Pythia WSC Score: 57.69. Result ID: 69a1592335dfcabb00e67393. [Reporting paper]() Recorded evaluation configuration: {"PS (billion)":"0.4","T (billion)":"300"}. This record does not assert independent reproduction. ### PEML WSC Score: 55.2. Result ID: 6a06cc5e47f34ce924ee0ee9. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"GLM-10B","Setting":"multitasking framework"}. This record does not assert independent reproduction. ### C-Poly WSC Score: 53.4. Result ID: 6a06cc5e47f34ce924ee0ee8. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"GLM-10B","Setting":"multitasking framework"}. This record does not assert independent reproduction. ### Poly WSC Score: 47. Result ID: 6a06cc5e47f34ce924ee0ee6. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"GLM-10B","Setting":"multitasking framework"}. This record does not assert independent reproduction. ### MHR WSC Score: 45.5. Result ID: 6a06cc5e47f34ce924ee0ee7. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"GLM-10B","Setting":"multitasking framework"}. This record does not assert independent reproduction. ### BLOOM WSC Score: 40.38. Result ID: 69a1592335dfcabb00e67394. [Reporting paper]() Recorded evaluation configuration: {"PS (billion)":"0.56","T (billion)":"350"}. This record does not assert independent reproduction. ### HGRN WSC Score: 40.38. Result ID: 69a1592335dfcabb00e6739a. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"PS (billion)":"1","T (billion)":"100"}. This record does not assert independent reproduction. ### MOE-LoRA WSC Score: 39.6. Result ID: 6a06cc5e47f34ce924ee0ee5. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"GLM-10B","Setting":"multitasking framework"}. This record does not assert independent reproduction. ### HGRN WSC Score: 38.46. Result ID: 69a1592335dfcabb00e67391. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"PS (billion)":"0.15","T (billion)":"100"}. This record does not assert independent reproduction. ### HGRN WSC Score: 38.46. Result ID: 69a1592335dfcabb00e67395. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"PS (billion)":"0.35","T (billion)":"100"}. This record does not assert independent reproduction. ### OPT WSC Score: 37.5. Result ID: 69a1592335dfcabb00e67397. [Reporting paper]() Recorded evaluation configuration: {"PS (billion)":"1.3","T (billion)":"300"}. This record does not assert independent reproduction. ### GPT-Neo WSC Score: 36.54. Result ID: 69a1592335dfcabb00e6738e. [Reporting paper]() Recorded evaluation configuration: {"PS (billion)":"0.13","T (billion)":"300"}. This record does not assert independent reproduction. ### OPT WSC Score: 36.54. Result ID: 69a1592335dfcabb00e6738f. [Reporting paper]() Recorded evaluation configuration: {"PS (billion)":"0.16","T (billion)":"300"}. This record does not assert independent reproduction. ### Pythia WSC Score: 36.54. Result ID: 69a1592335dfcabb00e67390. [Reporting paper]() Recorded evaluation configuration: {"PS (billion)":"0.16","T (billion)":"300"}. This record does not assert independent reproduction. ### OPT WSC Score: 36.54. Result ID: 69a1592335dfcabb00e67392. [Reporting paper]() Recorded evaluation configuration: {"PS (billion)":"0.35","T (billion)":"300"}. This record does not assert independent reproduction. ### GPT-Neo WSC Score: 36.54. Result ID: 69a1592335dfcabb00e67396. [Reporting paper]() Recorded evaluation configuration: {"PS (billion)":"1.3","T (billion)":"300"}. This record does not assert independent reproduction. ### Pythia WSC Score: 36.54. Result ID: 69a1592335dfcabb00e67398. [Reporting paper]() Recorded evaluation configuration: {"PS (billion)":"1.4","T (billion)":"300"}. This record does not assert independent reproduction. ### BLOOM WSC Score: 36.54. Result ID: 69a1592335dfcabb00e67399. [Reporting paper]() Recorded evaluation configuration: {"PS (billion)":"1.1","T (billion)":"350"}. This record does not assert independent reproduction. ### LoRA WSC Score: 32.3. Result ID: 6a06cc5e47f34ce924ee0ee4. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"GLM-10B","Setting":"multitasking framework"}. This record does not assert independent reproduction. Page 1 of 1. This is an index page, not the full collection.