# Natural Language Understanding on GLUE (test) (SST-2, QNLI, QQP, MNLI Subset) Source: [Natural Language Understanding on GLUE (test) (SST-2, QNLI, QQP, MNLI Subset)]() Collection index: [llms.txt]() Data updated: 2026-02-26T06:03:31.477Z Evaluation of general language understanding capabilities on the GLUE test set, specifically reporting scores for SST-2, QNLI, QQP, and MNLI tasks. A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. | Scope | Recorded value | | --- | --- | | Task | Natural Language Understanding | | Dataset (including any stated split) | GLUE (test) | | Metric | SST-2 Accuracy | | Unit | PERCENT | | Direction | Higher is better | | Results loaded | 33 of 33 | | Coverage | All results currently returned for this benchmark | Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol. ## Metrics - [SST-2 Accuracy](): Higher is better; unit: PERCENT. - [QNLI Accuracy](): Higher is better; unit: PERCENT. - [QQP Accuracy](): Higher is better; unit: PERCENT. - [MNLI Accuracy](): Higher is better; unit: PERCENT. - [Average Accuracy (GLUE)](): Higher is better; unit: PERCENT. ## Recorded results | Method | SST-2 Accuracy | Result date | Report paper | | --- | --- | --- | --- | | FedTT | 95.64 | Not recorded | 68fa79061e209131cea7bc39 | | LoRA | 94.32 | Not recorded | 68fa79061e209131cea7bc39 | | FFA-LoRA | 94.32 | Not recorded | 68fa79061e209131cea7bc39 | | FedTT | 93.96 | Not recorded | 68fa79061e209131cea7bc39 | | LA-LoRA | 93.94 | Not recorded | 699d30ae6017c4460ffab66b | | FedTT | 93.8 | Not recorded | 68fa79061e209131cea7bc39 | | FFA-LoRA | 93.73 | Not recorded | 68fa79061e209131cea7bc39 | | LoRA | 93.7 | Not recorded | 68fa79061e209131cea7bc39 | | RoLoRA | 93.65 | Not recorded | 699d30ae6017c4460ffab66b | | FFA-LoRA | 93.59 | Not recorded | 68fa79061e209131cea7bc39 | | LA-LoRA | 93.36 | Not recorded | 699d30ae6017c4460ffab66b | | LoRA | 93.32 | Not recorded | 68fa79061e209131cea7bc39 | | LA-LoRA | 93.12 | Not recorded | 699d30ae6017c4460ffab66b | | LA-LoRA | 93 | Not recorded | 699d30ae6017c4460ffab66b | | RoLoRA | 92.7 | Not recorded | 699d30ae6017c4460ffab66b | | LA-LoRA | 92.66 | Not recorded | 699d30ae6017c4460ffab66b | | FFA-LoRA | 92.56 | Not recorded | 699d30ae6017c4460ffab66b | | RoLoRA | 92.55 | Not recorded | 699d30ae6017c4460ffab66b | | FFA-LoRA | 92.53 | Not recorded | 699d30ae6017c4460ffab66b | | FFA-LoRA | 92.39 | Not recorded | 699d30ae6017c4460ffab66b | | DP-LoRA | 92.36 | Not recorded | 699d30ae6017c4460ffab66b | | FFA-LoRA | 92.32 | Not recorded | 699d30ae6017c4460ffab66b | | RoLoRA | 92.32 | Not recorded | 699d30ae6017c4460ffab66b | | DP-LoRA | 92.2 | Not recorded | 699d30ae6017c4460ffab66b | | RoLoRA | 92.12 | Not recorded | 699d30ae6017c4460ffab66b | | LA-LoRA | 92.11 | Not recorded | 699d30ae6017c4460ffab66b | | DP-LoRA | 92.07 | Not recorded | 699d30ae6017c4460ffab66b | | RoLoRA | 91.74 | Not recorded | 699d30ae6017c4460ffab66b | | DP-LoRA | 91.56 | Not recorded | 699d30ae6017c4460ffab66b | | FFA-LoRA | 91.06 | Not recorded | 699d30ae6017c4460ffab66b | | DP-LoRA | 90.71 | Not recorded | 699d30ae6017c4460ffab66b | | FFA-LoRA | 90.62 | Not recorded | 699d30ae6017c4460ffab66b | | DP-LoRA | 90.13 | Not recorded | 699d30ae6017c4460ffab66b | ### FedTT SST-2 Accuracy: 95.64. Result ID: 6995ba373badfaffe6bb867f. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Priv. Budget (ε)":1,"#Param.":"0.24M","Backbone":"RoBERTa-Large"}. This record does not assert independent reproduction. ### LoRA SST-2 Accuracy: 94.32. Result ID: 6995ba373badfaffe6bb867d. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Priv. Budget (ε)":1,"#Param.":"1.57M","rank":8,"Backbone":"RoBERTa-Large"}. This record does not assert independent reproduction. ### FFA-LoRA SST-2 Accuracy: 94.32. Result ID: 6995ba373badfaffe6bb867e. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Priv. Budget (ε)":1,"#Param.":"0.79M","rank":8,"Backbone":"RoBERTa-Large"}. This record does not assert independent reproduction. ### FedTT SST-2 Accuracy: 93.96. Result ID: 6995ba373badfaffe6bb867c. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Priv. Budget (ε)":3,"#Param.":"0.24M","Backbone":"RoBERTa-Large"}. This record does not assert independent reproduction. ### LA-LoRA SST-2 Accuracy: 93.94. Result ID: 699e58f85698665ec4199b23. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Privacy":"Non-private","Backbone":"RoBERTa-Base","Data Heterogeneity":"Dirichlet beta = 0.3"}. This record does not assert independent reproduction. ### FedTT SST-2 Accuracy: 93.8. Result ID: 6995ba373badfaffe6bb8679. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Priv. Budget (ε)":6,"#Param.":"0.24M","Backbone":"RoBERTa-Large"}. This record does not assert independent reproduction. ### FFA-LoRA SST-2 Accuracy: 93.73. Result ID: 6995ba373badfaffe6bb8678. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Priv. Budget (ε)":6,"#Param.":"0.79M","rank":8,"Backbone":"RoBERTa-Large"}. This record does not assert independent reproduction. ### LoRA SST-2 Accuracy: 93.7. Result ID: 6995ba373badfaffe6bb8677. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Priv. Budget (ε)":6,"#Param.":"1.57M","rank":8,"Backbone":"RoBERTa-Large"}. This record does not assert independent reproduction. ### RoLoRA SST-2 Accuracy: 93.65. Result ID: 699e58f85698665ec4199b22. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Privacy":"Non-private","Backbone":"RoBERTa-Base","Data Heterogeneity":"Dirichlet beta = 0.3"}. This record does not assert independent reproduction. ### FFA-LoRA SST-2 Accuracy: 93.59. Result ID: 6995ba373badfaffe6bb867b. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Priv. Budget (ε)":3,"#Param.":"0.79M","rank":8,"Backbone":"RoBERTa-Large"}. This record does not assert independent reproduction. ### LA-LoRA SST-2 Accuracy: 93.36. Result ID: 699e58f85698665ec4199b5b. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"Llama-2-7B","Privacy Epsilon (epsilon)":"1"}. This record does not assert independent reproduction. ### LoRA SST-2 Accuracy: 93.32. Result ID: 6995ba373badfaffe6bb867a. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Priv. Budget (ε)":3,"#Param.":"1.57M","rank":8,"Backbone":"RoBERTa-Large"}. This record does not assert independent reproduction. ### LA-LoRA SST-2 Accuracy: 93.12. Result ID: 699e58f85698665ec4199b17. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Privacy Budget (epsilon)":"3","Backbone":"RoBERTa-Base"}. This record does not assert independent reproduction. ### LA-LoRA SST-2 Accuracy: 93. Result ID: 699e58f85698665ec4199b1b. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Privacy Budget (epsilon)":"2","Backbone":"RoBERTa-Base"}. This record does not assert independent reproduction. ### RoLoRA SST-2 Accuracy: 92.7. Result ID: 699e58f85698665ec4199b16. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Privacy Budget (epsilon)":"3","Backbone":"RoBERTa-Base"}. This record does not assert independent reproduction. ### LA-LoRA SST-2 Accuracy: 92.66. Result ID: 699e58f85698665ec4199b1f. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Privacy Budget (epsilon)":"1","Backbone":"RoBERTa-Base"}. This record does not assert independent reproduction. ### FFA-LoRA SST-2 Accuracy: 92.56. Result ID: 699e58f85698665ec4199b21. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Privacy":"Non-private","Backbone":"RoBERTa-Base","Data Heterogeneity":"Dirichlet beta = 0.3"}. This record does not assert independent reproduction. ### RoLoRA SST-2 Accuracy: 92.55. Result ID: 699e58f85698665ec4199b1a. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Privacy Budget (epsilon)":"2","Backbone":"RoBERTa-Base"}. This record does not assert independent reproduction. ### FFA-LoRA SST-2 Accuracy: 92.53. Result ID: 699e58f85698665ec4199b59. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"Llama-2-7B","Privacy Epsilon (epsilon)":"1"}. This record does not assert independent reproduction. ### FFA-LoRA SST-2 Accuracy: 92.39. Result ID: 699e58f85698665ec4199b19. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Privacy Budget (epsilon)":"2","Backbone":"RoBERTa-Base"}. This record does not assert independent reproduction. ### DP-LoRA SST-2 Accuracy: 92.36. Result ID: 699e58f85698665ec4199b14. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Privacy Budget (epsilon)":"3","Backbone":"RoBERTa-Base"}. This record does not assert independent reproduction. ### FFA-LoRA SST-2 Accuracy: 92.32. Result ID: 699e58f85698665ec4199b15. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Privacy Budget (epsilon)":"3","Backbone":"RoBERTa-Base"}. This record does not assert independent reproduction. ### RoLoRA SST-2 Accuracy: 92.32. Result ID: 699e58f85698665ec4199b1e. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Privacy Budget (epsilon)":"1","Backbone":"RoBERTa-Base"}. This record does not assert independent reproduction. ### DP-LoRA SST-2 Accuracy: 92.2. Result ID: 699e58f85698665ec4199b18. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Privacy Budget (epsilon)":"2","Backbone":"RoBERTa-Base"}. This record does not assert independent reproduction. ### RoLoRA SST-2 Accuracy: 92.12. Result ID: 699e58f85698665ec4199b5a. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"Llama-2-7B","Privacy Epsilon (epsilon)":"1"}. This record does not assert independent reproduction. ### LA-LoRA SST-2 Accuracy: 92.11. Result ID: 699e58f85698665ec4199b27. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Privacy":"epsilon = 1","Backbone":"RoBERTa-Base","Data Heterogeneity":"Dirichlet beta = 0.3"}. This record does not assert independent reproduction. ### DP-LoRA SST-2 Accuracy: 92.07. Result ID: 699e58f85698665ec4199b20. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Privacy":"Non-private","Backbone":"RoBERTa-Base","Data Heterogeneity":"Dirichlet beta = 0.3"}. This record does not assert independent reproduction. ### RoLoRA SST-2 Accuracy: 91.74. Result ID: 699e58f85698665ec4199b26. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Privacy":"epsilon = 1","Backbone":"RoBERTa-Base","Data Heterogeneity":"Dirichlet beta = 0.3"}. This record does not assert independent reproduction. ### DP-LoRA SST-2 Accuracy: 91.56. Result ID: 699e58f85698665ec4199b58. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"Llama-2-7B","Privacy Epsilon (epsilon)":"1"}. This record does not assert independent reproduction. ### FFA-LoRA SST-2 Accuracy: 91.06. Result ID: 699e58f85698665ec4199b1d. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Privacy Budget (epsilon)":"1","Backbone":"RoBERTa-Base"}. This record does not assert independent reproduction. ### DP-LoRA SST-2 Accuracy: 90.71. Result ID: 699e58f85698665ec4199b1c. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Privacy Budget (epsilon)":"1","Backbone":"RoBERTa-Base"}. This record does not assert independent reproduction. ### FFA-LoRA SST-2 Accuracy: 90.62. Result ID: 699e58f85698665ec4199b25. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Privacy":"epsilon = 1","Backbone":"RoBERTa-Base","Data Heterogeneity":"Dirichlet beta = 0.3"}. This record does not assert independent reproduction. ### DP-LoRA SST-2 Accuracy: 90.13. Result ID: 699e58f85698665ec4199b24. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Privacy":"epsilon = 1","Backbone":"RoBERTa-Base","Data Heterogeneity":"Dirichlet beta = 0.3"}. This record does not assert independent reproduction. Page 1 of 1. This is an index page, not the full collection.