# Natural Language Understanding on GLUE (MNLI, QNLI, QQP, SST2 Subset) Source: [Natural Language Understanding on GLUE (MNLI, QNLI, QQP, SST2 Subset)]() Collection index: [llms.txt]() Data updated: 2026-05-11T07:20:21.197Z Evaluation of model performance across the specific subset of GLUE tasks: MNLI, QNLI, QQP, and SST-2. A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. | Scope | Recorded value | | --- | --- | | Task | Natural Language Understanding | | Dataset (including any stated split) | GLUE | | Metric | MNLI Accuracy | | Unit | PERCENT | | Direction | Higher is better | | Results loaded | 16 of 16 | | Coverage | All results currently returned for this benchmark | Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol. ## Metrics - [MNLI Accuracy](): Higher is better; unit: PERCENT. - [QNLI Accuracy](): Higher is better; unit: PERCENT. - [SST-2 Accuracy](): Higher is better; unit: PERCENT. - [QQP Accuracy](): Higher is better; unit: PERCENT. - [Average GLUE Score](): Higher is better; unit: PERCENT. - [MNLI-mm Accuracy](): Higher is better; unit: PERCENT. ## Recorded results | Method | MNLI Accuracy | Result date | Report paper | | --- | --- | --- | --- | | GLoRA | 85.59 | Not recorded | 6a016284dbc836b87d82300e | | GLoRA | 85.57 | Not recorded | 6a016284dbc836b87d82300e | | HetLoRA | 85.37 | Not recorded | 6a016284dbc836b87d82300e | | HetLoRA | 83.92 | Not recorded | 6a016284dbc836b87d82300e | | GLoRA | 83.02 | Not recorded | 6a016284dbc836b87d82300e | | FedTreeLoRA | 82.94 | Not recorded | 69b8e0060636e5bf4afbe98e | | HetLoRA | 82.74 | Not recorded | 6a016284dbc836b87d82300e | | FlexLoRA | 82.71 | Not recorded | 6a016284dbc836b87d82300e | | FlexLoRA | 81.89 | Not recorded | 6a016284dbc836b87d82300e | | FlexLoRA | 81.63 | Not recorded | 6a016284dbc836b87d82300e | | FedLEASE | 79.91 | Not recorded | 69b8e0060636e5bf4afbe98e | | FedDPA | 75 | Not recorded | 69b8e0060636e5bf4afbe98e | | FedSA | 74.29 | Not recorded | 69b8e0060636e5bf4afbe98e | | FedALT | 73.25 | Not recorded | 69b8e0060636e5bf4afbe98e | | FFA-LoRA | 72.17 | Not recorded | 69b8e0060636e5bf4afbe98e | | FedIT | 71.46 | Not recorded | 69b8e0060636e5bf4afbe98e | ### GLoRA MNLI Accuracy: 85.59. Result ID: 6a01832247f34ce924e2ed52. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Distribution":"Heavy-tail"}. This record does not assert independent reproduction. ### GLoRA MNLI Accuracy: 85.57. Result ID: 6a01832247f34ce924e2ed4c. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Distribution":"Normal"}. This record does not assert independent reproduction. ### HetLoRA MNLI Accuracy: 85.37. Result ID: 6a01832247f34ce924e2ed50. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Distribution":"Heavy-tail"}. This record does not assert independent reproduction. ### HetLoRA MNLI Accuracy: 83.92. Result ID: 6a01832247f34ce924e2ed4a. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Distribution":"Normal"}. This record does not assert independent reproduction. ### GLoRA MNLI Accuracy: 83.02. Result ID: 6a01832247f34ce924e2ed4f. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Distribution":"Uniform"}. This record does not assert independent reproduction. ### FedTreeLoRA MNLI Accuracy: 82.94. Result ID: 69b9026047f34ce924a11e59. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"number of clients":"16","clients per task":"4"}. This record does not assert independent reproduction. ### HetLoRA MNLI Accuracy: 82.74. Result ID: 6a01832247f34ce924e2ed4d. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Distribution":"Uniform"}. This record does not assert independent reproduction. ### FlexLoRA MNLI Accuracy: 82.71. Result ID: 6a01832247f34ce924e2ed51. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Distribution":"Heavy-tail"}. This record does not assert independent reproduction. ### FlexLoRA MNLI Accuracy: 81.89. Result ID: 6a01832247f34ce924e2ed4e. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Distribution":"Uniform"}. This record does not assert independent reproduction. ### FlexLoRA MNLI Accuracy: 81.63. Result ID: 6a01832247f34ce924e2ed4b. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Distribution":"Normal"}. This record does not assert independent reproduction. ### FedLEASE MNLI Accuracy: 79.91. Result ID: 69b9026047f34ce924a11e58. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"number of clients":"16","clients per task":"4"}. This record does not assert independent reproduction. ### FedDPA MNLI Accuracy: 75. Result ID: 69b9026047f34ce924a11e56. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"number of clients":"16","clients per task":"4"}. This record does not assert independent reproduction. ### FedSA MNLI Accuracy: 74.29. Result ID: 69b9026047f34ce924a11e55. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"number of clients":"16","clients per task":"4"}. This record does not assert independent reproduction. ### FedALT MNLI Accuracy: 73.25. Result ID: 69b9026047f34ce924a11e57. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"number of clients":"16","clients per task":"4"}. This record does not assert independent reproduction. ### FFA-LoRA MNLI Accuracy: 72.17. Result ID: 69b9026047f34ce924a11e54. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"number of clients":"16","clients per task":"4"}. This record does not assert independent reproduction. ### FedIT MNLI Accuracy: 71.46. Result ID: 69b9026047f34ce924a11e53. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"number of clients":"16","clients per task":"4"}. This record does not assert independent reproduction. Page 1 of 1. This is an index page, not the full collection.