# Natural Language Understanding on GLUE i.i.d. distribution (val) Source: [Natural Language Understanding on GLUE i.i.d. distribution (val)]() Collection index: [llms.txt]() Data updated: 2026-02-26T05:50:18.714Z Evaluation of general language understanding capabilities across the GLUE tasks, specifically using the i.i.d. validation distribution split. A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. | Scope | Recorded value | | --- | --- | | Task | Natural Language Understanding | | Dataset (including any stated split) | GLUE i.i.d. distribution (val) | | Metric | MRPC | | Unit | PERCENT | | Direction | Higher is better | | Results loaded | 8 of 8 | | Coverage | All results currently returned for this benchmark | Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol. ## Metrics - [MRPC](): Higher is better; unit: PERCENT. - [SST-2](): Higher is better; unit: PERCENT. - [QNLI](): Higher is better; unit: PERCENT. - [QQP](): Higher is better; unit: PERCENT. - [MNLI](): Higher is better; unit: PERCENT. - [GLUE Avg Acc](): Higher is better; unit: PERCENT. ## Recorded results | Method | MRPC | Result date | Report paper | | --- | --- | --- | --- | | FedTT | 0.9268 | Not recorded | 68fa79061e209131cea7bc39 | | FedTT+ | 0.926 | Not recorded | 68fa79061e209131cea7bc39 | | LoRA | 0.9187 | Not recorded | 68fa79061e209131cea7bc39 | | LoRA | 0.9172 | Not recorded | 68fa79061e209131cea7bc39 | | BitFit | 0.9133 | Not recorded | 68fa79061e209131cea7bc39 | | RoLoRA | 0.9117 | Not recorded | 68fa79061e209131cea7bc39 | | Prompt | 0.8296 | Not recorded | 68fa79061e209131cea7bc39 | | P-Tuning | 0.8201 | Not recorded | 68fa79061e209131cea7bc39 | ### FedTT MRPC: 0.9268. Result ID: 6995ba373badfaffe6bb8675. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"DeBERTa-Base","# Param.":"0.06M","FL Setting":"cross-silo","Distribution":"i.i.d."}. This record does not assert independent reproduction. ### FedTT+ MRPC: 0.926. Result ID: 6995ba373badfaffe6bb8676. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"DeBERTa-Base","# Param.":"0.02M","FL Setting":"cross-silo","Distribution":"i.i.d."}. This record does not assert independent reproduction. ### LoRA MRPC: 0.9187. Result ID: 6995ba373badfaffe6bb866f. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"DeBERTa-Base","# Param.":"0.30M","Rank":"r=8","FL Setting":"cross-silo","Distribution":"i.i.d."}. This record does not assert independent reproduction. ### LoRA MRPC: 0.9172. Result ID: 6995ba373badfaffe6bb8671. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"DeBERTa-Base","# Param.":"0.15M","Rank":"r=4","FL Setting":"cross-silo","Distribution":"i.i.d."}. This record does not assert independent reproduction. ### BitFit MRPC: 0.9133. Result ID: 6995ba373badfaffe6bb8672. [Reporting paper]() Recorded evaluation configuration: {"Backbone":"DeBERTa-Base","# Param.":"0.10M","FL Setting":"cross-silo","Distribution":"i.i.d."}. This record does not assert independent reproduction. ### RoLoRA MRPC: 0.9117. Result ID: 6995ba373badfaffe6bb8673. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"DeBERTa-Base","# Param.":"0.08M","Rank":"r=4","FL Setting":"cross-silo","Distribution":"i.i.d."}. This record does not assert independent reproduction. ### Prompt MRPC: 0.8296. Result ID: 6995ba373badfaffe6bb8674. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"DeBERTa-Base","# Param.":"0.01M","FL Setting":"cross-silo","Distribution":"i.i.d."}. This record does not assert independent reproduction. ### P-Tuning MRPC: 0.8201. Result ID: 6995ba373badfaffe6bb8670. [Reporting paper]() Recorded evaluation configuration: {"Backbone":"DeBERTa-Base","# Param.":"0.30M","FL Setting":"cross-silo","Distribution":"i.i.d."}. This record does not assert independent reproduction. Page 1 of 1. This is an index page, not the full collection.