# Natural Language Understanding on GLUE & SuperGLUE (test) Source: [Natural Language Understanding on GLUE & SuperGLUE (test)]() Collection index: [llms.txt]() Data updated: 2026-04-21T12:58:38.774Z Evaluation of general language understanding capabilities across the constituent tasks of the combined GLUE and SuperGLUE benchmark suites, reporting results on the test split. A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. | Scope | Recorded value | | --- | --- | | Task | Natural Language Understanding | | Dataset (including any stated split) | GLUE & SuperGLUE (test) | | Metric | RTE Accuracy | | Unit | PERCENT | | Direction | Higher is better | | Results loaded | 21 of 21 | | Coverage | All results currently returned for this benchmark | Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol. ## Metrics - [RTE Accuracy](): Higher is better; unit: PERCENT. - [STS-B Score](): Higher is better; unit: PERCENT. - [MRPC Accuracy](): Higher is better; unit: PERCENT. - [CoLA MCC](): Higher is better; unit: PERCENT. - [MultiRC F1](): Higher is better; unit: PERCENT. - [BoolQ Accuracy](): Higher is better; unit: PERCENT. - [WiC Accuracy](): Higher is better; unit: PERCENT. - [WSC Accuracy](): Higher is better; unit: PERCENT. - [CB Score](): Higher is better; unit: PERCENT. - [Average Score (GLUE/SuperGLUE)](): Higher is better; unit: PERCENT. - [SST2 Accuracy](): Higher is better; unit: PERCENT. - [COPA Accuracy](): Higher is better; unit: PERCENT. ## Recorded results | Method | RTE Accuracy | Result date | Report paper | | --- | --- | --- | --- | | PEMT | 83 | Not recorded | 68fa738b1e209131cea5d1bc | | MPT | 79.4 | Not recorded | 68fa738b1e209131cea5d1bc | | FO | 78 | Not recorded | 697846b5a2b28e761d87c634 | | ConMeZO | 75.1 | Not recorded | 690ada065fbd5403840047d3 | | LoRA | 74.1 | Not recorded | 68fa738b1e209131cea5d1bc | | AGZO | 73.6 | Not recorded | 697846b5a2b28e761d87c634 | | ATTEMPT | 73.4 | Not recorded | 68fa738b1e209131cea5d1bc | | MEZO | 72.9 | Not recorded | 697846b5a2b28e761d87c634 | | FT | 71.9 | Not recorded | 68fa738b1e209131cea5d1bc | | Adapter | 71.9 | Not recorded | 68fa738b1e209131cea5d1bc | | AdaMix | 70.5 | Not recorded | 68fa738b1e209131cea5d1bc | | SPoT | 69.8 | Not recorded | 68fa738b1e209131cea5d1bc | | LOZO | 69.7 | Not recorded | 697846b5a2b28e761d87c634 | | ICL | 68.2 | Not recorded | 697846b5a2b28e761d87c634 | | BitFit | 67.6 | Not recorded | 68fa738b1e209131cea5d1bc | | MixDA | 66.9 | Not recorded | 68fa738b1e209131cea5d1bc | | Zero | 58.1 | Not recorded | 697846b5a2b28e761d87c634 | | PT | 54.7 | Not recorded | 68fa738b1e209131cea5d1bc | | HiZOO | 54.2 | Not recorded | 690ada065fbd5403840047d3 | ### PEMT RTE Accuracy: 83. Result ID: 6992ef273a6c31484912768e. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Backbone":"T5-Base","Tuning Mode":"Knowledge Transfer PEFT"}. This record does not assert independent reproduction. ### MPT RTE Accuracy: 79.4. Result ID: 6992ef273a6c31484912768b. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"T5-Base","Tuning Mode":"Prompt-based tuning"}. This record does not assert independent reproduction. ### FO RTE Accuracy: 78. Result ID: 69991dd25698665ec484b15b. [Reporting paper]() Recorded evaluation configuration: {"Method Type":"First-Order Fine-tuning","Model":"Pangu-1B"}. This record does not assert independent reproduction. ### ConMeZO RTE Accuracy: 75.1. Result ID: 69e7746747f34ce924cf3881. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"RoBERTa-Large","Seeds":"5 seeds","Hyperparameter Selection":"default configuration"}. This record does not assert independent reproduction. ### LoRA RTE Accuracy: 74.1. Result ID: 6992ef273a6c314849127688. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Backbone":"T5-Base","Tuning Mode":"Parameter-efficient tuning"}. This record does not assert independent reproduction. ### AGZO RTE Accuracy: 73.6. Result ID: 69991dd25698665ec484b15c. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Method Type":"Zeroth-Order Fine-tuning","Model":"Pangu-1B"}. This record does not assert independent reproduction. ### ATTEMPT RTE Accuracy: 73.4. Result ID: 6992ef273a6c31484912768a. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"T5-Base","Tuning Mode":"Prompt-based tuning"}. This record does not assert independent reproduction. ### MEZO RTE Accuracy: 72.9. Result ID: 69991dd25698665ec484b15d. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Method Type":"Zeroth-Order Fine-tuning","Model":"Pangu-1B"}. This record does not assert independent reproduction. ### FT RTE Accuracy: 71.9. Result ID: 6992ef273a6c314849127684. [Reporting paper]() Recorded evaluation configuration: {"Backbone":"T5-Base","Tuning Mode":"Full fine-tuning"}. This record does not assert independent reproduction. ### Adapter RTE Accuracy: 71.9. Result ID: 6992ef273a6c314849127687. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Backbone":"T5-Base","Tuning Mode":"Vanilla adapter-based tuning"}. This record does not assert independent reproduction. ### AdaMix RTE Accuracy: 70.5. Result ID: 6992ef273a6c31484912768d. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"T5-Base","Tuning Mode":"Adapter-based tuning"}. This record does not assert independent reproduction. ### SPoT RTE Accuracy: 69.8. Result ID: 6992ef273a6c314849127689. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Backbone":"T5-Base","Tuning Mode":"Prompt-based tuning"}. This record does not assert independent reproduction. ### LOZO RTE Accuracy: 69.7. Result ID: 69991dd25698665ec484b15e. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Method Type":"Zeroth-Order Fine-tuning","Model":"Pangu-1B"}. This record does not assert independent reproduction. ### ICL RTE Accuracy: 68.2. Result ID: 69991dd25698665ec484b160. [Reporting paper]() Recorded evaluation configuration: {"Evaluation Protocol":"In-context learning","Model":"Pangu-1B"}. This record does not assert independent reproduction. ### BitFit RTE Accuracy: 67.6. Result ID: 6992ef273a6c314849127686. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Backbone":"T5-Base","Tuning Mode":"Parameter-efficient tuning"}. This record does not assert independent reproduction. ### MixDA RTE Accuracy: 66.9. Result ID: 6992ef273a6c31484912768c. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Backbone":"T5-Base","Tuning Mode":"Adapter-based tuning"}. This record does not assert independent reproduction. ### Zero RTE Accuracy: 58.1. Result ID: 69991dd25698665ec484b15f. [Reporting paper]() Recorded evaluation configuration: {"Evaluation Protocol":"Zero-shot prompting","Model":"Pangu-1B"}. This record does not assert independent reproduction. ### PT RTE Accuracy: 54.7. Result ID: 6992ef273a6c314849127685. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"T5-Base","Tuning Mode":"Vanilla prompt tuning"}. This record does not assert independent reproduction. ### HiZOO RTE Accuracy: 54.2. Result ID: 69e7746747f34ce924cf3880. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"RoBERTa-Large","Seeds":"5 seeds","Hyperparameter Selection":"best across learning rates {10^-5, 10^-6, 10^-7}"}. This record does not assert independent reproduction. Page 1 of 1. This is an index page, not the full collection.