# Natural Language Understanding on GLUE (MNLI, QNLI, SST2, QQP) Source: [Natural Language Understanding on GLUE (MNLI, QNLI, SST2, QQP)]() Collection index: [llms.txt]() Data updated: 2026-03-17T07:30:02.737Z Evaluation of model performance across the specific subset of GLUE tasks: MNLI, QNLI, SST-2, and QQP. A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. | Scope | Recorded value | | --- | --- | | Task | Natural Language Understanding | | Dataset (including any stated split) | GLUE | | Metric | MNLI Accuracy | | Unit | PERCENT | | Direction | Higher is better | | Results loaded | 8 of 8 | | Coverage | All results currently returned for this benchmark | Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol. ## Metrics - [MNLI Accuracy](): Higher is better; unit: PERCENT. - [QNLI Accuracy](): Higher is better; unit: PERCENT. - [SST-2 Accuracy](): Higher is better; unit: PERCENT. - [QQP Accuracy](): Higher is better; unit: PERCENT. - [Average GLUE Accuracy](): Higher is better; unit: PERCENT. - [Performance Delta](): Higher is better; unit: SCALAR. ## Recorded results | Method | MNLI Accuracy | Result date | Report paper | | --- | --- | --- | --- | | FedTreeLoRA | 88.15 | Not recorded | 69b8e0060636e5bf4afbe98e | | FedLEASE | 86.21 | Not recorded | 69b8e0060636e5bf4afbe98e | | FedALT | 84.03 | Not recorded | 69b8e0060636e5bf4afbe98e | | FedDPA | 83.97 | Not recorded | 69b8e0060636e5bf4afbe98e | | FedSA | 83.63 | Not recorded | 69b8e0060636e5bf4afbe98e | | FedIT | 83.18 | Not recorded | 69b8e0060636e5bf4afbe98e | | FFA-LoRA | 83.13 | Not recorded | 69b8e0060636e5bf4afbe98e | | FFA-LoRA | 83.02 | Not recorded | 69b8e0060636e5bf4afbe98e | ### FedTreeLoRA MNLI Accuracy: 88.15. Result ID: 69b9026047f34ce924a11e61. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Rank":"4","% Param":"0.1107%"}. This record does not assert independent reproduction. ### FedLEASE MNLI Accuracy: 86.21. Result ID: 69b9026047f34ce924a11e60. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Rank":"4","% Param":"0.1521%"}. This record does not assert independent reproduction. ### FedALT MNLI Accuracy: 84.03. Result ID: 69b9026047f34ce924a11e5f. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Rank":"4","% Param":"0.1383%"}. This record does not assert independent reproduction. ### FedDPA MNLI Accuracy: 83.97. Result ID: 69b9026047f34ce924a11e5e. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Rank":"4","% Param":"0.1107%"}. This record does not assert independent reproduction. ### FedSA MNLI Accuracy: 83.63. Result ID: 69b9026047f34ce924a11e5d. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Rank":"4","% Param":"0.1107%"}. This record does not assert independent reproduction. ### FedIT MNLI Accuracy: 83.18. Result ID: 69b9026047f34ce924a11e5a. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Rank":"4","% Param":"0.1107%"}. This record does not assert independent reproduction. ### FFA-LoRA MNLI Accuracy: 83.13. Result ID: 69b9026047f34ce924a11e5c. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Rank":"8","% Param":"0.1107%"}. This record does not assert independent reproduction. ### FFA-LoRA MNLI Accuracy: 83.02. Result ID: 69b9026047f34ce924a11e5b. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Rank":"4","% Param":"0.0553%"}. This record does not assert independent reproduction. Page 1 of 1. This is an index page, not the full collection.