# Natural Language Understanding on GLUE (SST-2 and MNLI Subset) Source: [Natural Language Understanding on GLUE (SST-2 and MNLI Subset)]() Collection index: [llms.txt]() Data updated: 2026-03-13T08:42:44.086Z Evaluation of model performance specifically on the SST-2 and MNLI components of the GLUE benchmark suite. A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. | Scope | Recorded value | | --- | --- | | Task | Natural Language Understanding | | Dataset (including any stated split) | GLUE | | Metric | SST-2 Accuracy | | Unit | PERCENT | | Direction | Higher is better | | Results loaded | 17 of 17 | | Coverage | All results currently returned for this benchmark | Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol. ## Metrics - [SST-2 Accuracy](): Higher is better; unit: PERCENT. - [MNLI Accuracy](): Higher is better; unit: PERCENT. - [GLUE Average Accuracy](): Higher is better; unit: PERCENT. ## Recorded results | Method | SST-2 Accuracy | Result date | Report paper | | --- | --- | --- | --- | | ModernBERT large | 97.1 | Not recorded | 68fa7a111e209131cea80ee7 | | DeBERTa-v1-xl | 97.1 | Not recorded | 68fa7a111e209131cea80ee7 | | Ettin-Enc-1B | 97.1 | Not recorded | 68fa7a111e209131cea80ee7 | | Ettin-Enc-400m | 96.7 | Not recorded | 68fa7a111e209131cea80ee7 | | ModernBERT base | 96 | Not recorded | 68fa7a111e209131cea80ee7 | | Ettin-Enc-150m | 95.8 | Not recorded | 68fa7a111e209131cea80ee7 | | Ettin-Enc-68m | 94.4 | Not recorded | 68fa7a111e209131cea80ee7 | | MiniLM L12 | 93.3 | Not recorded | 68fa7a111e209131cea80ee7 | | BERT large | 93.3 | Not recorded | 68fa7a111e209131cea80ee7 | | DistilRoBERTa | 93.1 | Not recorded | 68fa7a111e209131cea80ee7 | | BERT base | 93.1 | Not recorded | 68fa7a111e209131cea80ee7 | | DistilBERT | 92.2 | Not recorded | 68fa7a111e209131cea80ee7 | | Ettin-Enc-32m | 92 | Not recorded | 68fa7a111e209131cea80ee7 | | TinyBERT | 91.2 | Not recorded | 68fa7a111e209131cea80ee7 | | Ettin-Enc-17m | 91.2 | Not recorded | 68fa7a111e209131cea80ee7 | | BERT-small | 90.1 | Not recorded | 68fa7a111e209131cea80ee7 | | BERT-mini | 88.3 | Not recorded | 68fa7a111e209131cea80ee7 | ### ModernBERT large SST-2 Accuracy: 97.1. Result ID: 69b3cdf447f34ce9249df29b. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model Size Category":"Large","Parameters":"353-395M"}. This record does not assert independent reproduction. ### DeBERTa-v1-xl SST-2 Accuracy: 97.1. Result ID: 69b3cdf447f34ce9249df29d. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Model Size Category":"XL","Parameters":"884M-1.2B"}. This record does not assert independent reproduction. ### Ettin-Enc-1B SST-2 Accuracy: 97.1. Result ID: 69b3cdf447f34ce9249df29e. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model Size Category":"XL","Parameters":"884M-1.2B"}. This record does not assert independent reproduction. ### Ettin-Enc-400m SST-2 Accuracy: 96.7. Result ID: 69b3cdf447f34ce9249df29c. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model Size Category":"Large","Parameters":"353-395M"}. This record does not assert independent reproduction. ### ModernBERT base SST-2 Accuracy: 96. Result ID: 69b3cdf447f34ce9249df298. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model Size Category":"Base","Parameters":"123-150M"}. This record does not assert independent reproduction. ### Ettin-Enc-150m SST-2 Accuracy: 95.8. Result ID: 69b3cdf447f34ce9249df299. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model Size Category":"Base","Parameters":"123-150M"}. This record does not assert independent reproduction. ### Ettin-Enc-68m SST-2 Accuracy: 94.4. Result ID: 69b3cdf447f34ce9249df296. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model Size Category":"Small","Parameters":"68-82M"}. This record does not assert independent reproduction. ### MiniLM L12 SST-2 Accuracy: 93.3. Result ID: 69b3cdf447f34ce9249df292. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model Size Category":"XS","Parameters":"28-33M"}. This record does not assert independent reproduction. ### BERT large SST-2 Accuracy: 93.3. Result ID: 69b3cdf447f34ce9249df29a. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Model Size Category":"Large","Parameters":"353-395M"}. This record does not assert independent reproduction. ### DistilRoBERTa SST-2 Accuracy: 93.1. Result ID: 69b3cdf447f34ce9249df295. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Model Size Category":"Small","Parameters":"68-82M"}. This record does not assert independent reproduction. ### BERT base SST-2 Accuracy: 93.1. Result ID: 69b3cdf447f34ce9249df297. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Model Size Category":"Base","Parameters":"123-150M"}. This record does not assert independent reproduction. ### DistilBERT SST-2 Accuracy: 92.2. Result ID: 69b3cdf447f34ce9249df294. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Model Size Category":"Small","Parameters":"68-82M"}. This record does not assert independent reproduction. ### Ettin-Enc-32m SST-2 Accuracy: 92. Result ID: 69b3cdf447f34ce9249df293. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model Size Category":"XS","Parameters":"28-33M"}. This record does not assert independent reproduction. ### TinyBERT SST-2 Accuracy: 91.2. Result ID: 69b3cdf447f34ce9249df28f. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Model Size Category":"XXS","Parameters":"7-17M"}. This record does not assert independent reproduction. ### Ettin-Enc-17m SST-2 Accuracy: 91.2. Result ID: 69b3cdf447f34ce9249df290. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model Size Category":"XXS","Parameters":"7-17M"}. This record does not assert independent reproduction. ### BERT-small SST-2 Accuracy: 90.1. Result ID: 69b3cdf447f34ce9249df291. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model Size Category":"XS","Parameters":"28-33M"}. This record does not assert independent reproduction. ### BERT-mini SST-2 Accuracy: 88.3. Result ID: 69b3cdf447f34ce9249df28e. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model Size Category":"XXS","Parameters":"7-17M"}. This record does not assert independent reproduction. Page 1 of 1. This is an index page, not the full collection.