# Natural Language Understanding on GLUE (8-Task Subset: CoLA, SST-2, MRPC, QQP, MNLI, QNLI, RTE, STS-B)

Source: [Natural Language Understanding on GLUE (8-Task Subset: CoLA, SST-2, MRPC, QQP, MNLI, QNLI, RTE, STS-B)](<https://www.sota2.com/research/sota/natural-language-understanding-on-glue-8-task-subset-cola-sst-2-mrpc-qqp-mnli-qnli-rte-sts-b>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2026-05-07T07:15:48.809Z

Evaluation of model performance across eight specified subtasks of the GLUE benchmark suite: CoLA, SST-2, MRPC, QQP, MNLI, QNLI, RTE, and STS-B.

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

| Scope | Recorded value |
| --- | --- |
| Task | Natural Language Understanding |
| Dataset (including any stated split) | GLUE |
| Metric | CoLA Accuracy |
| Unit | PERCENT |
| Direction | Higher is better |
| Results loaded | 15 of 15 |
| Coverage | All results currently returned for this benchmark |

Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol.

## Metrics

- [CoLA Accuracy](<https://www.sota2.com/research/sota/natural-language-understanding-on-glue-8-task-subset-cola-sst-2-mrpc-qqp-mnli-qnli-rte-sts-b.md?metric=CoLA%20Accuracy>): Higher is better; unit: PERCENT.

- [SST-2 Accuracy](<https://www.sota2.com/research/sota/natural-language-understanding-on-glue-8-task-subset-cola-sst-2-mrpc-qqp-mnli-qnli-rte-sts-b.md?metric=SST-2%20Accuracy>): Higher is better; unit: PERCENT.

- [MRPC Accuracy](<https://www.sota2.com/research/sota/natural-language-understanding-on-glue-8-task-subset-cola-sst-2-mrpc-qqp-mnli-qnli-rte-sts-b.md?metric=MRPC%20Accuracy>): Higher is better; unit: PERCENT.

- [QQP Accuracy](<https://www.sota2.com/research/sota/natural-language-understanding-on-glue-8-task-subset-cola-sst-2-mrpc-qqp-mnli-qnli-rte-sts-b.md?metric=QQP%20Accuracy>): Higher is better; unit: PERCENT.

- [MNLI Accuracy](<https://www.sota2.com/research/sota/natural-language-understanding-on-glue-8-task-subset-cola-sst-2-mrpc-qqp-mnli-qnli-rte-sts-b.md?metric=MNLI%20Accuracy>): Higher is better; unit: PERCENT.

- [QNLI Accuracy](<https://www.sota2.com/research/sota/natural-language-understanding-on-glue-8-task-subset-cola-sst-2-mrpc-qqp-mnli-qnli-rte-sts-b.md?metric=QNLI%20Accuracy>): Higher is better; unit: PERCENT.

- [RTE Accuracy](<https://www.sota2.com/research/sota/natural-language-understanding-on-glue-8-task-subset-cola-sst-2-mrpc-qqp-mnli-qnli-rte-sts-b.md?metric=RTE%20Accuracy>): Higher is better; unit: PERCENT.

- [STS-B Score](<https://www.sota2.com/research/sota/natural-language-understanding-on-glue-8-task-subset-cola-sst-2-mrpc-qqp-mnli-qnli-rte-sts-b.md?metric=STS-B%20Score>): Higher is better; unit: PERCENT.

- [Average GLUE Score (8 Tasks)](<https://www.sota2.com/research/sota/natural-language-understanding-on-glue-8-task-subset-cola-sst-2-mrpc-qqp-mnli-qnli-rte-sts-b.md?metric=Average%20GLUE%20Score%20(8%20Tasks)>): Higher is better; unit: PERCENT.

## Recorded results

| Method | CoLA Accuracy | Result date | Report paper |
| --- | --- | --- | --- |
| MP-ISMoE† (T5-large) | 66.7 | Not recorded | 69fc1c7d5497b3b670c8845a |
| MP-ISMoE‡ (T5-large) | 66.4 | Not recorded | 69fc1c7d5497b3b670c8845a |
| UniPT (T5-large) | 65.7 | Not recorded | 69fc1c7d5497b3b670c8845a |
| SHERL (T5-large) | 65.6 | Not recorded | 69fc1c7d5497b3b670c8845a |
| LST (T5-large) | 65.3 | Not recorded | 69fc1c7d5497b3b670c8845a |
| Adapter | 64.4 | Not recorded | 69fc1c7d5497b3b670c8845a |
| MP-ISMoE† | 63.4 | Not recorded | 69fc1c7d5497b3b670c8845a |
| LoRA | 63.3 | Not recorded | 69fc1c7d5497b3b670c8845a |
| Fully-FT | 62.8 | Not recorded | 69fc1c7d5497b3b670c8845a |
| MP-ISMoE‡ | 62.4 | Not recorded | 69fc1c7d5497b3b670c8845a |
| UniPT | 62.2 | Not recorded | 69fc1c7d5497b3b670c8845a |
| BitFit | 61.8 | Not recorded | 69fc1c7d5497b3b670c8845a |
| SHERL | 61.1 | Not recorded | 69fc1c7d5497b3b670c8845a |
| LST | 58.1 | Not recorded | 69fc1c7d5497b3b670c8845a |
| Prompt | 0 | Not recorded | 69fc1c7d5497b3b670c8845a |

### MP-ISMoE† (T5-large)

CoLA Accuracy: 66.7. Result ID: 69fc3c0f47f34ce924de8c79.

[Reporting paper](<https://www.sota2.com/research/paper/69fc1c7d5497b3b670c8845a.md>)

[Method origin paper](<https://www.sota2.com/research/paper/69fc1c7d5497b3b670c8845a.md>)

Recorded evaluation configuration: {"Backbone":"T5-large","Params (%)":"1.75","Memory Train (G)":"9.9","Memory Test (G)":"2.82"}. This record does not assert independent reproduction.

### MP-ISMoE‡ (T5-large)

CoLA Accuracy: 66.4. Result ID: 69fc3c0f47f34ce924de8c7a.

[Reporting paper](<https://www.sota2.com/research/paper/69fc1c7d5497b3b670c8845a.md>)

[Method origin paper](<https://www.sota2.com/research/paper/69fc1c7d5497b3b670c8845a.md>)

Recorded evaluation configuration: {"Backbone":"T5-large","Params (%)":"0.81","Memory Train (G)":"7.6","Memory Test (G)":"2.80"}. This record does not assert independent reproduction.

### UniPT (T5-large)

CoLA Accuracy: 65.7. Result ID: 69fc3c0f47f34ce924de8c77.

[Reporting paper](<https://www.sota2.com/research/paper/69fc1c7d5497b3b670c8845a.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa72711e209131cea56572.md>)

[Implementation](<https://github.com/Paranioar/UniPT>)

Recorded evaluation configuration: {"Backbone":"T5-large","Params (%)":"0.92","Memory Train (G)":"9.1","Memory Test (G)":"2.82"}. This record does not assert independent reproduction.

### SHERL (T5-large)

CoLA Accuracy: 65.6. Result ID: 69fc3c0f47f34ce924de8c78.

[Reporting paper](<https://www.sota2.com/research/paper/69fc1c7d5497b3b670c8845a.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa74121e209131cea5f9e9.md>)

Recorded evaluation configuration: {"Backbone":"T5-large","Params (%)":"0.64","Memory Train (G)":"7.1","Memory Test (G)":"2.80"}. This record does not assert independent reproduction.

### LST (T5-large)

CoLA Accuracy: 65.3. Result ID: 69fc3c0f47f34ce924de8c76.

[Reporting paper](<https://www.sota2.com/research/paper/69fc1c7d5497b3b670c8845a.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa6dc81e209131cea3c2d3.md>)

Recorded evaluation configuration: {"Backbone":"T5-large","Params (%)":"1.23","Memory Train (G)":"12.2","Memory Test (G)":"2.88"}. This record does not assert independent reproduction.

### Adapter

CoLA Accuracy: 64.4. Result ID: 69fc3c0f47f34ce924de8c6d.

[Reporting paper](<https://www.sota2.com/research/paper/69fc1c7d5497b3b670c8845a.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa65e31e209131cea0d324.md>)

[Implementation](<https://github.com/google-research/adapter-bert>)

Recorded evaluation configuration: {"Backbone":"T5-base","Params (%)":"1.63","Memory Train (G)":"13.0","Memory Test (G)":"0.87"}. This record does not assert independent reproduction.

### MP-ISMoE†

CoLA Accuracy: 63.4. Result ID: 69fc3c0f47f34ce924de8c74.

[Reporting paper](<https://www.sota2.com/research/paper/69fc1c7d5497b3b670c8845a.md>)

[Method origin paper](<https://www.sota2.com/research/paper/69fc1c7d5497b3b670c8845a.md>)

Recorded evaluation configuration: {"Backbone":"T5-base","Params (%)":"2.41","Memory Train (G)":"3.2","Memory Test (G)":"0.86"}. This record does not assert independent reproduction.

### LoRA

CoLA Accuracy: 63.3. Result ID: 69fc3c0f47f34ce924de8c6e.

[Reporting paper](<https://www.sota2.com/research/paper/69fc1c7d5497b3b670c8845a.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa6b121e209131cea2bf59.md>)

[Implementation](<https://github.com/microsoft/LoRA>)

Recorded evaluation configuration: {"Backbone":"T5-base","Params (%)":"1.71","Memory Train (G)":"12.6","Memory Test (G)":"0.86"}. This record does not assert independent reproduction.

### Fully-FT

CoLA Accuracy: 62.8. Result ID: 69fc3c0f47f34ce924de8c6c.

[Reporting paper](<https://www.sota2.com/research/paper/69fc1c7d5497b3b670c8845a.md>)

Recorded evaluation configuration: {"Backbone":"T5-base","Params (%)":"100","Memory Train (G)":"17.6","Memory Test (G)":"0.86"}. This record does not assert independent reproduction.

### MP-ISMoE‡

CoLA Accuracy: 62.4. Result ID: 69fc3c0f47f34ce924de8c75.

[Reporting paper](<https://www.sota2.com/research/paper/69fc1c7d5497b3b670c8845a.md>)

[Method origin paper](<https://www.sota2.com/research/paper/69fc1c7d5497b3b670c8845a.md>)

Recorded evaluation configuration: {"Backbone":"T5-base","Params (%)":"1.48","Memory Train (G)":"3.3","Memory Test (G)":"0.87"}. This record does not assert independent reproduction.

### UniPT

CoLA Accuracy: 62.2. Result ID: 69fc3c0f47f34ce924de8c72.

[Reporting paper](<https://www.sota2.com/research/paper/69fc1c7d5497b3b670c8845a.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa72711e209131cea56572.md>)

[Implementation](<https://github.com/Paranioar/UniPT>)

Recorded evaluation configuration: {"Backbone":"T5-base","Params (%)":"1.36","Memory Train (G)":"2.9","Memory Test (G)":"0.86"}. This record does not assert independent reproduction.

### BitFit

CoLA Accuracy: 61.8. Result ID: 69fc3c0f47f34ce924de8c6f.

[Reporting paper](<https://www.sota2.com/research/paper/69fc1c7d5497b3b670c8845a.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa6d5a1e209131cea39632.md>)

[Implementation](<https://www.github.com/benzakenelad/BitFit>)

Recorded evaluation configuration: {"Backbone":"T5-base","Params (%)":"0.13","Memory Train (G)":"10.7","Memory Test (G)":"0.86"}. This record does not assert independent reproduction.

### SHERL

CoLA Accuracy: 61.1. Result ID: 69fc3c0f47f34ce924de8c73.

[Reporting paper](<https://www.sota2.com/research/paper/69fc1c7d5497b3b670c8845a.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa74121e209131cea5f9e9.md>)

Recorded evaluation configuration: {"Backbone":"T5-base","Params (%)":"0.85","Memory Train (G)":"2.9","Memory Test (G)":"0.87"}. This record does not assert independent reproduction.

### LST

CoLA Accuracy: 58.1. Result ID: 69fc3c0f47f34ce924de8c71.

[Reporting paper](<https://www.sota2.com/research/paper/69fc1c7d5497b3b670c8845a.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa6dc81e209131cea3c2d3.md>)

Recorded evaluation configuration: {"Backbone":"T5-base","Params (%)":"1.74","Memory Train (G)":"5.5","Memory Test (G)":"0.88"}. This record does not assert independent reproduction.

### Prompt

CoLA Accuracy: 0. Result ID: 69fc3c0f47f34ce924de8c70.

[Reporting paper](<https://www.sota2.com/research/paper/69fc1c7d5497b3b670c8845a.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa690f1e209131cea1fe0e.md>)

Recorded evaluation configuration: {"Backbone":"T5-base","Params (%)":"0.03","Memory Train (G)":"22.2","Memory Test (G)":"0.87"}. This record does not assert independent reproduction.

Page 1 of 1. This is an index page, not the full collection.
