# Natural Language Understanding on GLUE8 (val)

Source: [Natural Language Understanding on GLUE8 (val)](<https://www.sota2.com/research/sota/natural-language-understanding-on-glue8-val>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2026-06-02T09:38:00.186Z

Evaluation of general language understanding capabilities across the 8 core GLUE tasks (CoLA, SST-2, MRPC, STS-B, QQP, MNLI, QNLI, RTE) reported on the validation split.

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

| Scope | Recorded value |
| --- | --- |
| Task | Natural Language Understanding |
| Dataset (including any stated split) | GLUE8 (val) |
| Metric | CoLA Score |
| Unit | SCALAR |
| Direction | Higher is better |
| Results loaded | 18 of 18 |
| Coverage | All results currently returned for this benchmark |

Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol.

## Metrics

- [CoLA Score](<https://www.sota2.com/research/sota/natural-language-understanding-on-glue8-val.md?metric=CoLA%20Score>): Higher is better; unit: SCALAR.

- [SST-2 Accuracy](<https://www.sota2.com/research/sota/natural-language-understanding-on-glue8-val.md?metric=SST-2%20Accuracy>): Higher is better; unit: SCALAR.

- [MRPC Score](<https://www.sota2.com/research/sota/natural-language-understanding-on-glue8-val.md?metric=MRPC%20Score>): Higher is better; unit: SCALAR.

- [STS-B Correlation](<https://www.sota2.com/research/sota/natural-language-understanding-on-glue8-val.md?metric=STS-B%20Correlation>): Higher is better; unit: SCALAR.

- [QQP Score](<https://www.sota2.com/research/sota/natural-language-understanding-on-glue8-val.md?metric=QQP%20Score>): Higher is better; unit: SCALAR.

- [MNLI Accuracy](<https://www.sota2.com/research/sota/natural-language-understanding-on-glue8-val.md?metric=MNLI%20Accuracy>): Higher is better; unit: SCALAR.

- [QNLI Accuracy](<https://www.sota2.com/research/sota/natural-language-understanding-on-glue8-val.md?metric=QNLI%20Accuracy>): Higher is better; unit: SCALAR.

- [RTE Accuracy](<https://www.sota2.com/research/sota/natural-language-understanding-on-glue8-val.md?metric=RTE%20Accuracy>): Higher is better; unit: SCALAR.

- [Average GLUE Score](<https://www.sota2.com/research/sota/natural-language-understanding-on-glue8-val.md?metric=Average%20GLUE%20Score>): Higher is better; unit: SCALAR.

## Recorded results

| Method | CoLA Score | Result date | Report paper |
| --- | --- | --- | --- |
| LORA+ | 0.578 | Not recorded | 6a1e6395dbd7150506f8523b |
| RITE | 0.515 | Not recorded | 6a1e6395dbd7150506f8523b |
| ADAMW | 0.504 | Not recorded | 6a1e6395dbd7150506f8523b |
| LAMB | 0.468 | Not recorded | 6a1e6395dbd7150506f8523b |
| FFA | 0.456 | Not recorded | 6a1e6395dbd7150506f8523b |
| PRISM | 0.444 | Not recorded | 6a1e6395dbd7150506f8523b |
| LORA+ | 0.436 | Not recorded | 6a1e6395dbd7150506f8523b |
| LORA+ | 0.434 | Not recorded | 6a1e6395dbd7150506f8523b |
| LAMB | 0.414 | Not recorded | 6a1e6395dbd7150506f8523b |
| ADAMW | 0.41 | Not recorded | 6a1e6395dbd7150506f8523b |
| ADAMW | 0.407 | Not recorded | 6a1e6395dbd7150506f8523b |
| PRISM | 0.406 | Not recorded | 6a1e6395dbd7150506f8523b |
| LAMB | 0.396 | Not recorded | 6a1e6395dbd7150506f8523b |
| PRISM | 0.392 | Not recorded | 6a1e6395dbd7150506f8523b |
| FFA | 0.355 | Not recorded | 6a1e6395dbd7150506f8523b |
| FFA | 0.337 | Not recorded | 6a1e6395dbd7150506f8523b |
| RITE | 0.235 | Not recorded | 6a1e6395dbd7150506f8523b |
| RITE | 0.221 | Not recorded | 6a1e6395dbd7150506f8523b |

### LORA+

CoLA Score: 0.578. Result ID: 6a1ea462fbc373592c192afa.

[Reporting paper](<https://www.sota2.com/research/paper/6a1e6395dbd7150506f8523b.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa74061e209131cea5f4d8.md>)

Recorded evaluation configuration: {"SETTING":"NON-DP","Backbone":"Gemma-3-4B-pt"}. This record does not assert independent reproduction.

### RITE

CoLA Score: 0.515. Result ID: 6a1ea462fbc373592c192af8.

[Reporting paper](<https://www.sota2.com/research/paper/6a1e6395dbd7150506f8523b.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa7a091e209131cea80b04.md>)

Recorded evaluation configuration: {"SETTING":"NON-DP","Backbone":"Gemma-3-4B-pt"}. This record does not assert independent reproduction.

### ADAMW

CoLA Score: 0.504. Result ID: 6a1ea462fbc373592c192af9.

[Reporting paper](<https://www.sota2.com/research/paper/6a1e6395dbd7150506f8523b.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa55761e209131ce9946db.md>)

Recorded evaluation configuration: {"SETTING":"NON-DP","Backbone":"Gemma-3-4B-pt"}. This record does not assert independent reproduction.

### LAMB

CoLA Score: 0.468. Result ID: 6a1ea462fbc373592c192afb.

[Reporting paper](<https://www.sota2.com/research/paper/6a1e6395dbd7150506f8523b.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa66e71e209131cea13099.md>)

[Implementation](<https://github.com/tensorflow/addons/blob/master/tensorflow_addons/optimizers/lamb.py>)

Recorded evaluation configuration: {"SETTING":"NON-DP","Backbone":"Gemma-3-4B-pt"}. This record does not assert independent reproduction.

### FFA

CoLA Score: 0.456. Result ID: 6a1ea462fbc373592c192af7.

[Reporting paper](<https://www.sota2.com/research/paper/6a1e6395dbd7150506f8523b.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa728f1e209131cea57138.md>)

Recorded evaluation configuration: {"SETTING":"NON-DP","Backbone":"Gemma-3-4B-pt"}. This record does not assert independent reproduction.

### PRISM

CoLA Score: 0.444. Result ID: 6a1ea462fbc373592c192b02.

[Reporting paper](<https://www.sota2.com/research/paper/6a1e6395dbd7150506f8523b.md>)

[Method origin paper](<https://www.sota2.com/research/paper/6a1e6395dbd7150506f8523b.md>)

Recorded evaluation configuration: {"SETTING":"ε = 6","Backbone":"Gemma-3-4B-pt","δ":"10^-5"}. This record does not assert independent reproduction.

### LORA+

CoLA Score: 0.436. Result ID: 6a1ea462fbc373592c192b00.

[Reporting paper](<https://www.sota2.com/research/paper/6a1e6395dbd7150506f8523b.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa74061e209131cea5f4d8.md>)

Recorded evaluation configuration: {"SETTING":"ε = 6","Backbone":"Gemma-3-4B-pt","δ":"10^-5"}. This record does not assert independent reproduction.

### LORA+

CoLA Score: 0.434. Result ID: 6a1ea462fbc373592c192b06.

[Reporting paper](<https://www.sota2.com/research/paper/6a1e6395dbd7150506f8523b.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa74061e209131cea5f4d8.md>)

Recorded evaluation configuration: {"SETTING":"ε = 3","Backbone":"Gemma-3-4B-pt","δ":"10^-5"}. This record does not assert independent reproduction.

### LAMB

CoLA Score: 0.414. Result ID: 6a1ea462fbc373592c192b01.

[Reporting paper](<https://www.sota2.com/research/paper/6a1e6395dbd7150506f8523b.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa66e71e209131cea13099.md>)

[Implementation](<https://github.com/tensorflow/addons/blob/master/tensorflow_addons/optimizers/lamb.py>)

Recorded evaluation configuration: {"SETTING":"ε = 6","Backbone":"Gemma-3-4B-pt","δ":"10^-5"}. This record does not assert independent reproduction.

### ADAMW

CoLA Score: 0.41. Result ID: 6a1ea462fbc373592c192b05.

[Reporting paper](<https://www.sota2.com/research/paper/6a1e6395dbd7150506f8523b.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa55761e209131ce9946db.md>)

Recorded evaluation configuration: {"SETTING":"ε = 3","Backbone":"Gemma-3-4B-pt","δ":"10^-5"}. This record does not assert independent reproduction.

### ADAMW

CoLA Score: 0.407. Result ID: 6a1ea462fbc373592c192aff.

[Reporting paper](<https://www.sota2.com/research/paper/6a1e6395dbd7150506f8523b.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa55761e209131ce9946db.md>)

Recorded evaluation configuration: {"SETTING":"ε = 6","Backbone":"Gemma-3-4B-pt","δ":"10^-5"}. This record does not assert independent reproduction.

### PRISM

CoLA Score: 0.406. Result ID: 6a1ea462fbc373592c192b08.

[Reporting paper](<https://www.sota2.com/research/paper/6a1e6395dbd7150506f8523b.md>)

[Method origin paper](<https://www.sota2.com/research/paper/6a1e6395dbd7150506f8523b.md>)

Recorded evaluation configuration: {"SETTING":"ε = 3","Backbone":"Gemma-3-4B-pt","δ":"10^-5"}. This record does not assert independent reproduction.

### LAMB

CoLA Score: 0.396. Result ID: 6a1ea462fbc373592c192b07.

[Reporting paper](<https://www.sota2.com/research/paper/6a1e6395dbd7150506f8523b.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa66e71e209131cea13099.md>)

[Implementation](<https://github.com/tensorflow/addons/blob/master/tensorflow_addons/optimizers/lamb.py>)

Recorded evaluation configuration: {"SETTING":"ε = 3","Backbone":"Gemma-3-4B-pt","δ":"10^-5"}. This record does not assert independent reproduction.

### PRISM

CoLA Score: 0.392. Result ID: 6a1ea462fbc373592c192afc.

[Reporting paper](<https://www.sota2.com/research/paper/6a1e6395dbd7150506f8523b.md>)

[Method origin paper](<https://www.sota2.com/research/paper/6a1e6395dbd7150506f8523b.md>)

Recorded evaluation configuration: {"SETTING":"NON-DP","Backbone":"Gemma-3-4B-pt"}. This record does not assert independent reproduction.

### FFA

CoLA Score: 0.355. Result ID: 6a1ea462fbc373592c192afd.

[Reporting paper](<https://www.sota2.com/research/paper/6a1e6395dbd7150506f8523b.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa728f1e209131cea57138.md>)

Recorded evaluation configuration: {"SETTING":"ε = 6","Backbone":"Gemma-3-4B-pt","δ":"10^-5"}. This record does not assert independent reproduction.

### FFA

CoLA Score: 0.337. Result ID: 6a1ea462fbc373592c192b03.

[Reporting paper](<https://www.sota2.com/research/paper/6a1e6395dbd7150506f8523b.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa728f1e209131cea57138.md>)

Recorded evaluation configuration: {"SETTING":"ε = 3","Backbone":"Gemma-3-4B-pt","δ":"10^-5"}. This record does not assert independent reproduction.

### RITE

CoLA Score: 0.235. Result ID: 6a1ea462fbc373592c192afe.

[Reporting paper](<https://www.sota2.com/research/paper/6a1e6395dbd7150506f8523b.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa7a091e209131cea80b04.md>)

Recorded evaluation configuration: {"SETTING":"ε = 6","Backbone":"Gemma-3-4B-pt","δ":"10^-5"}. This record does not assert independent reproduction.

### RITE

CoLA Score: 0.221. Result ID: 6a1ea462fbc373592c192b04.

[Reporting paper](<https://www.sota2.com/research/paper/6a1e6395dbd7150506f8523b.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa7a091e209131cea80b04.md>)

Recorded evaluation configuration: {"SETTING":"ε = 3","Backbone":"Gemma-3-4B-pt","δ":"10^-5"}. This record does not assert independent reproduction.

Page 1 of 1. This is an index page, not the full collection.