# Natural Language Understanding on GLUE (val test) Source: [Natural Language Understanding on GLUE (val test)]() Collection index: [llms.txt]() Data updated: 2026-02-26T06:03:31.182Z Evaluation of general language understanding capabilities across a specific subset of GLUE tasks (SST-2, RTE, MRPC, QQP, MNLI, QNLI) reported on the official validation/test split. A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. | Scope | Recorded value | | --- | --- | | Task | Natural Language Understanding | | Dataset (including any stated split) | GLUE official (val test) | | Metric | SST-2 Accuracy | | Unit | PERCENT | | Direction | Higher is better | | Results loaded | 18 of 18 | | Coverage | All results currently returned for this benchmark | Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol. ## Metrics - [SST-2 Accuracy](): Higher is better; unit: PERCENT. - [RTE Accuracy](): Higher is better; unit: PERCENT. - [MRPC Accuracy](): Higher is better; unit: PERCENT. - [QQP Accuracy](): Higher is better; unit: PERCENT. - [MNLI Accuracy](): Higher is better; unit: PERCENT. - [QNLI Accuracy](): Higher is better; unit: PERCENT. ## Recorded results | Method | SST-2 Accuracy | Result date | Report paper | | --- | --- | --- | --- | | XLNet | 0.97 | Not recorded | 68fa6f461e209131cea44b05 | | ELECTRA | 0.969 | Not recorded | 68fa6f461e209131cea44b05 | | UNIMO | 0.968 | Not recorded | 68fa6f461e209131cea44b05 | | DeBERTa | 0.968 | Not recorded | 68fa6f461e209131cea44b05 | | OFA | 0.966 | Not recorded | 68fa6f461e209131cea44b05 | | RoBERTa | 0.964 | Not recorded | 68fa6f461e209131cea44b05 | | mPLUG-2Deberta | 0.962 | Not recorded | 68fa6f461e209131cea44b05 | | mPLUG-2 | 0.951 | Not recorded | 68fa6f461e209131cea44b05 | | mPLUG-2Base | 0.935 | Not recorded | 68fa6f461e209131cea44b05 | | BERT | 0.932 | Not recorded | 68fa6f461e209131cea44b05 | | SimVLM | 0.909 | Not recorded | 68fa6f461e209131cea44b05 | | FLAVA | 0.909 | Not recorded | 68fa6f461e209131cea44b05 | | ViLBERT | 0.904 | Not recorded | 68fa6f461e209131cea44b05 | | LXMERT | 0.902 | Not recorded | 68fa6f461e209131cea44b05 | | Uni-Perceiver | 0.902 | Not recorded | 68fa6f461e209131cea44b05 | | VL-BERT | 0.898 | Not recorded | 68fa6f461e209131cea44b05 | | UNITER | 0.897 | Not recorded | 68fa6f461e209131cea44b05 | | VisualBERT | 0.894 | Not recorded | 68fa6f461e209131cea44b05 | ### XLNet SST-2 Accuracy: 0.97. Result ID: 69909f0a3a6c31484901edb0. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Pre-training Strategy":"Natural Language"}. This record does not assert independent reproduction. ### ELECTRA SST-2 Accuracy: 0.969. Result ID: 69909f0a3a6c31484901edb1. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Pre-training Strategy":"Natural Language"}. This record does not assert independent reproduction. ### UNIMO SST-2 Accuracy: 0.968. Result ID: 69909f0a3a6c31484901edac. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Pre-training Strategy":"Multimodal"}. This record does not assert independent reproduction. ### DeBERTa SST-2 Accuracy: 0.968. Result ID: 69909f0a3a6c31484901edb2. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Pre-training Strategy":"Natural Language"}. This record does not assert independent reproduction. ### OFA SST-2 Accuracy: 0.966. Result ID: 69909f0a3a6c31484901edad. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Pre-training Strategy":"Multimodal"}. This record does not assert independent reproduction. ### RoBERTa SST-2 Accuracy: 0.964. Result ID: 69909f0a3a6c31484901edaf. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Pre-training Strategy":"Natural Language"}. This record does not assert independent reproduction. ### mPLUG-2Deberta SST-2 Accuracy: 0.962. Result ID: 69909f0a3a6c31484901edb5. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: Not recorded. This record does not assert independent reproduction. ### mPLUG-2 SST-2 Accuracy: 0.951. Result ID: 69909f0a3a6c31484901edb4. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: Not recorded. This record does not assert independent reproduction. ### mPLUG-2Base SST-2 Accuracy: 0.935. Result ID: 69909f0a3a6c31484901edb3. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: Not recorded. This record does not assert independent reproduction. ### BERT SST-2 Accuracy: 0.932. Result ID: 69909f0a3a6c31484901edae. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Pre-training Strategy":"Natural Language"}. This record does not assert independent reproduction. ### SimVLM SST-2 Accuracy: 0.909. Result ID: 69909f0a3a6c31484901edaa. [Reporting paper]() Recorded evaluation configuration: {"Pre-training Strategy":"Multimodal"}. This record does not assert independent reproduction. ### FLAVA SST-2 Accuracy: 0.909. Result ID: 69909f0a3a6c31484901edab. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Pre-training Strategy":"Multimodal"}. This record does not assert independent reproduction. ### ViLBERT SST-2 Accuracy: 0.904. Result ID: 69909f0a3a6c31484901eda7. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Pre-training Strategy":"Multimodal"}. This record does not assert independent reproduction. ### LXMERT SST-2 Accuracy: 0.902. Result ID: 69909f0a3a6c31484901eda8. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Pre-training Strategy":"Multimodal"}. This record does not assert independent reproduction. ### Uni-Perceiver SST-2 Accuracy: 0.902. Result ID: 69909f0a3a6c31484901eda9. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Pre-training Strategy":"Multimodal"}. This record does not assert independent reproduction. ### VL-BERT SST-2 Accuracy: 0.898. Result ID: 69909f0a3a6c31484901eda6. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Pre-training Strategy":"Multimodal"}. This record does not assert independent reproduction. ### UNITER SST-2 Accuracy: 0.897. Result ID: 69909f0a3a6c31484901eda5. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Pre-training Strategy":"Multimodal"}. This record does not assert independent reproduction. ### VisualBERT SST-2 Accuracy: 0.894. Result ID: 69909f0a3a6c31484901eda4. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Pre-training Strategy":"Multimodal"}. This record does not assert independent reproduction. Page 1 of 1. This is an index page, not the full collection.