# Natural Language Understanding on GLUE (test) (CoLA, SST-2, MRPC, QQP, MNLI, QNLI, RTE) Source: [Natural Language Understanding on GLUE (test) (CoLA, SST-2, MRPC, QQP, MNLI, QNLI, RTE)]() Collection index: [llms.txt]() Data updated: 2026-05-01T09:03:41.147Z Evaluation of general language understanding capabilities on the GLUE test set, reporting scores for CoLA, SST-2, MRPC, QQP, MNLI, QNLI, and RTE tasks, along with the overall average. A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. | Scope | Recorded value | | --- | --- | | Task | Natural Language Understanding | | Dataset (including any stated split) | GLUE (test) | | Metric | CoLA Score | | Unit | SCALAR | | Direction | Higher is better | | Results loaded | 18 of 18 | | Coverage | All results currently returned for this benchmark | Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol. ## Metrics - [CoLA Score](): Higher is better; unit: SCALAR. - [SST-2 Accuracy](): Higher is better; unit: SCALAR. - [MRPC Score](): Higher is better; unit: SCALAR. - [QQP Score](): Higher is better; unit: SCALAR. - [MNLI Accuracy](): Higher is better; unit: SCALAR. - [QNLI Accuracy](): Higher is better; unit: SCALAR. - [RTE Accuracy](): Higher is better; unit: SCALAR. - [GLUE Average Score](): Higher is better; unit: SCALAR. ## Recorded results | Method | CoLA Score | Result date | Report paper | | --- | --- | --- | --- | | Individual | 0.5203 | Not recorded | 69f433a355a3c32af89fd217 | | Traditional MTL | 0.5168 | Not recorded | 69f433a355a3c32af89fd217 | | Auto-FlexSwitch+MSE | 0.5119 | Not recorded | 69f433a355a3c32af89fd217 | | Twin-merging | 0.5084 | Not recorded | 69f433a355a3c32af89fd217 | | Auto-FlexSwitch+CKA | 0.5075 | Not recorded | 69f433a355a3c32af89fd217 | | Auto-FlexSwitch+KL | 0.5068 | Not recorded | 69f433a355a3c32af89fd217 | | Auto-Switch | 0.495 | Not recorded | 69f433a355a3c32af89fd217 | | EMR-merging | 0.4891 | Not recorded | 69f433a355a3c32af89fd217 | | AdaMerging++ | 0.2917 | Not recorded | 69f433a355a3c32af89fd217 | | Ties-Merging | 0.2015 | Not recorded | 69f433a355a3c32af89fd217 | | RegMean | 0.1947 | Not recorded | 69f433a355a3c32af89fd217 | | AdaMerging | 0.1566 | Not recorded | 69f433a355a3c32af89fd217 | | Task-Arithmetic | 0.1516 | Not recorded | 69f433a355a3c32af89fd217 | | DF-Merge | 0.1423 | Not recorded | 69f433a355a3c32af89fd217 | | DARE | 0.1384 | Not recorded | 69f433a355a3c32af89fd217 | | Fisher Merging | 0.098 | Not recorded | 69f433a355a3c32af89fd217 | | Weight-Averaging | 0.0352 | Not recorded | 69f433a355a3c32af89fd217 | | Pre-trained | -0.0454 | Not recorded | 69f433a355a3c32af89fd217 | ### Individual CoLA Score: 0.5203. Result ID: 69f46bd447f34ce924da9142. [Reporting paper]() Recorded evaluation configuration: {"Backbone":"Mamba-130M"}. This record does not assert independent reproduction. ### Traditional MTL CoLA Score: 0.5168. Result ID: 69f46bd447f34ce924da9141. [Reporting paper]() Recorded evaluation configuration: {"Backbone":"Mamba-130M"}. This record does not assert independent reproduction. ### Auto-FlexSwitch+MSE CoLA Score: 0.5119. Result ID: 69f46bd447f34ce924da9151. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Type":"Dynamic","Storage (MB)":"23.90","Backbone":"Mamba-130M","lambda":"0.9"}. This record does not assert independent reproduction. ### Twin-merging CoLA Score: 0.5084. Result ID: 69f46bd447f34ce924da914d. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Type":"Dynamic","Storage (MB)":"795.29","Backbone":"Mamba-130M"}. This record does not assert independent reproduction. ### Auto-FlexSwitch+CKA CoLA Score: 0.5075. Result ID: 69f46bd447f34ce924da914f. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Type":"Dynamic","Storage (MB)":"61.60","Backbone":"Mamba-130M","lambda":"50"}. This record does not assert independent reproduction. ### Auto-FlexSwitch+KL CoLA Score: 0.5068. Result ID: 69f46bd447f34ce924da9150. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Type":"Dynamic","Storage (MB)":"34.96","Backbone":"Mamba-130M","lambda":"5"}. This record does not assert independent reproduction. ### Auto-Switch CoLA Score: 0.495. Result ID: 69f46bd447f34ce924da914e. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Type":"Dynamic","Storage (MB)":"216.46","Backbone":"Mamba-130M"}. This record does not assert independent reproduction. ### EMR-merging CoLA Score: 0.4891. Result ID: 69f46bd447f34ce924da914c. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Type":"Dynamic","Storage (MB)":"600.37","Backbone":"Mamba-130M"}. This record does not assert independent reproduction. ### AdaMerging++ CoLA Score: 0.2917. Result ID: 69f46bd447f34ce924da914b. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Type":"Static","Backbone":"Mamba-130M"}. This record does not assert independent reproduction. ### Ties-Merging CoLA Score: 0.2015. Result ID: 69f46bd447f34ce924da9146. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Type":"Static","Backbone":"Mamba-130M"}. This record does not assert independent reproduction. ### RegMean CoLA Score: 0.1947. Result ID: 69f46bd447f34ce924da9149. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Type":"Static","Backbone":"Mamba-130M"}. This record does not assert independent reproduction. ### AdaMerging CoLA Score: 0.1566. Result ID: 69f46bd447f34ce924da914a. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Type":"Static","Backbone":"Mamba-130M"}. This record does not assert independent reproduction. ### Task-Arithmetic CoLA Score: 0.1516. Result ID: 69f46bd447f34ce924da9144. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Type":"Static","Backbone":"Mamba-130M"}. This record does not assert independent reproduction. ### DF-Merge CoLA Score: 0.1423. Result ID: 69f46bd447f34ce924da9148. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Type":"Static","Backbone":"Mamba-130M"}. This record does not assert independent reproduction. ### DARE CoLA Score: 0.1384. Result ID: 69f46bd447f34ce924da9145. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Type":"Static","Backbone":"Mamba-130M"}. This record does not assert independent reproduction. ### Fisher Merging CoLA Score: 0.098. Result ID: 69f46bd447f34ce924da9147. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Type":"Static","Backbone":"Mamba-130M"}. This record does not assert independent reproduction. ### Weight-Averaging CoLA Score: 0.0352. Result ID: 69f46bd447f34ce924da9143. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Type":"Static","Backbone":"Mamba-130M"}. This record does not assert independent reproduction. ### Pre-trained CoLA Score: -0.0454. Result ID: 69f46bd447f34ce924da9140. [Reporting paper]() Recorded evaluation configuration: {"Backbone":"Mamba-130M"}. This record does not assert independent reproduction. Page 1 of 1. This is an index page, not the full collection.