# Text Classification on NLU Tasks (SST-2, MR, Subj, SST-5, MPQA) Source: [Text Classification on NLU Tasks (SST-2, MR, Subj, SST-5, MPQA)]() Collection index: [llms.txt]() Data updated: 2026-06-15T07:10:52.731Z Evaluation of Text Classification performance across the specific subset of NLU tasks: SST-2, MR, Subj, SST-5, and MPQA, reporting individual and average accuracy. A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. | Scope | Recorded value | | --- | --- | | Task | Text Classification | | Dataset (including any stated split) | NLU Tasks (SST-2, MR, Subj, SST-5, MPQA) | | Metric | SST-2 Accuracy | | Unit | PERCENT | | Direction | Higher is better | | Results loaded | 29 of 29 | | Coverage | All results currently returned for this benchmark | Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol. ## Metrics - [SST-2 Accuracy](): Higher is better; unit: PERCENT. - [MR Accuracy](): Higher is better; unit: PERCENT. - [Subj Accuracy](): Higher is better; unit: PERCENT. - [SST-5 Accuracy](): Higher is better; unit: PERCENT. - [MPQA Accuracy](): Higher is better; unit: PERCENT. - [Average Accuracy](): Higher is better; unit: PERCENT. ## Recorded results | Method | SST-2 Accuracy | Result date | Report paper | | --- | --- | --- | --- | | KGERMAR | 95.4 | Not recorded | 6a2f8723d33dfb23e783c076 | | KGERMAR | 95.4 | Not recorded | 6a2f8723d33dfb23e783c076 | | ERMAR | 94.7 | Not recorded | 68fa79d81e209131cea7fb95 | | ERMAR | 94.7 | Not recorded | 6a2f8723d33dfb23e783c076 | | KGERMAR | 94.3 | Not recorded | 6a2f8723d33dfb23e783c076 | | LongLLaMA | 94.1 | Not recorded | 68fa79d81e209131cea7fb95 | | LongLLaMA | 94.1 | Not recorded | 6a2f8723d33dfb23e783c076 | | ERMAR | 93.6 | Not recorded | 68fa79d81e209131cea7fb95 | | ERMAR | 93.6 | Not recorded | 68fa79d81e209131cea7fb95 | | OpenLLaMA | 93.6 | Not recorded | 68fa79d81e209131cea7fb95 | | ERMAR | 93.6 | Not recorded | 6a2f8723d33dfb23e783c076 | | ERMAR | 93.6 | Not recorded | 6a2f8723d33dfb23e783c076 | | OpenLLaMA | 93.6 | Not recorded | 6a2f8723d33dfb23e783c076 | | MemLong | 93.5 | Not recorded | 68fa79d81e209131cea7fb95 | | MemLong | 93.5 | Not recorded | 6a2f8723d33dfb23e783c076 | | OpenLLaMA w./ Rag | 92.2 | Not recorded | 68fa79d81e209131cea7fb95 | | OpenLLaMA w/ RAG | 92.2 | Not recorded | 6a2f8723d33dfb23e783c076 | | MemLong | 91.5 | Not recorded | 68fa79d81e209131cea7fb95 | | MemLong | 91.5 | Not recorded | 6a2f8723d33dfb23e783c076 | | LongLLaMA | 91.4 | Not recorded | 68fa79d81e209131cea7fb95 | | LongLLaMA | 91.4 | Not recorded | 6a2f8723d33dfb23e783c076 | | MemLong | 91 | Not recorded | 68fa79d81e209131cea7fb95 | | MemLong | 91 | Not recorded | 6a2f8723d33dfb23e783c076 | | OpenLLaMA w./ Rag | 90.9 | Not recorded | 68fa79d81e209131cea7fb95 | | OpenLLaMA w/ RAG | 90.9 | Not recorded | 6a2f8723d33dfb23e783c076 | | OpenLLaMA | 90.7 | Not recorded | 68fa79d81e209131cea7fb95 | | OpenLLaMA | 90.7 | Not recorded | 6a2f8723d33dfb23e783c076 | | LongLLaMA | 90.4 | Not recorded | 68fa79d81e209131cea7fb95 | | LongLLaMA | 90.4 | Not recorded | 6a2f8723d33dfb23e783c076 | ### KGERMAR SST-2 Accuracy: 95.4. Result ID: 6a2fa563fbc373592c2b78a2. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"In-Context (In-C)":"4","In-Memory (In-M)":"4"}. This record does not assert independent reproduction. ### KGERMAR SST-2 Accuracy: 95.4. Result ID: 6a2fa563fbc373592c2b78a6. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"In-Context (In-C)":"4","In-Memory (In-M)":"18"}. This record does not assert independent reproduction. ### ERMAR SST-2 Accuracy: 94.7. Result ID: 6a0c3826fbc373592c02e7f4. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"In-Context (InC)":"20","In-Memory (InM)":"18"}. This record does not assert independent reproduction. ### ERMAR SST-2 Accuracy: 94.7. Result ID: 6a2fa563fbc373592c2b78ab. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"In-Context (In-C)":"20","In-Memory (In-M)":"18"}. This record does not assert independent reproduction. ### KGERMAR SST-2 Accuracy: 94.3. Result ID: 6a2fa563fbc373592c2b78ac. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"In-Context (In-C)":"20","In-Memory (In-M)":"18"}. This record does not assert independent reproduction. ### LongLLaMA SST-2 Accuracy: 94.1. Result ID: 6a0c3826fbc373592c02e7f2. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"In-Context (InC)":"20","In-Memory (InM)":"18"}. This record does not assert independent reproduction. ### LongLLaMA SST-2 Accuracy: 94.1. Result ID: 6a2fa563fbc373592c2b78a9. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"In-Context (In-C)":"20","In-Memory (In-M)":"18"}. This record does not assert independent reproduction. ### ERMAR SST-2 Accuracy: 93.6. Result ID: 6a0c3826fbc373592c02e7ec. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"In-Context (InC)":"4","In-Memory (InM)":"4"}. This record does not assert independent reproduction. ### ERMAR SST-2 Accuracy: 93.6. Result ID: 6a0c3826fbc373592c02e7ef. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"In-Context (InC)":"4","In-Memory (InM)":"18"}. This record does not assert independent reproduction. ### OpenLLaMA SST-2 Accuracy: 93.6. Result ID: 6a0c3826fbc373592c02e7f0. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"In-Context (InC)":"20","In-Memory (InM)":"N/A"}. This record does not assert independent reproduction. ### ERMAR SST-2 Accuracy: 93.6. Result ID: 6a2fa563fbc373592c2b78a1. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"In-Context (In-C)":"4","In-Memory (In-M)":"4"}. This record does not assert independent reproduction. ### ERMAR SST-2 Accuracy: 93.6. Result ID: 6a2fa563fbc373592c2b78a5. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"In-Context (In-C)":"4","In-Memory (In-M)":"18"}. This record does not assert independent reproduction. ### OpenLLaMA SST-2 Accuracy: 93.6. Result ID: 6a2fa563fbc373592c2b78a7. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"In-Context (In-C)":"20","In-Memory (In-M)":"N/A"}. This record does not assert independent reproduction. ### MemLong SST-2 Accuracy: 93.5. Result ID: 6a0c3826fbc373592c02e7f3. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"In-Context (InC)":"20","In-Memory (InM)":"18"}. This record does not assert independent reproduction. ### MemLong SST-2 Accuracy: 93.5. Result ID: 6a2fa563fbc373592c2b78aa. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"In-Context (In-C)":"20","In-Memory (In-M)":"18"}. This record does not assert independent reproduction. ### OpenLLaMA w./ Rag SST-2 Accuracy: 92.2. Result ID: 6a0c3826fbc373592c02e7f1. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"In-Context (InC)":"20","In-Memory (InM)":"18"}. This record does not assert independent reproduction. ### OpenLLaMA w/ RAG SST-2 Accuracy: 92.2. Result ID: 6a2fa563fbc373592c2b78a8. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"In-Context (In-C)":"20","In-Memory (In-M)":"18"}. This record does not assert independent reproduction. ### MemLong SST-2 Accuracy: 91.5. Result ID: 6a0c3826fbc373592c02e7eb. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"In-Context (InC)":"4","In-Memory (InM)":"4"}. This record does not assert independent reproduction. ### MemLong SST-2 Accuracy: 91.5. Result ID: 6a2fa563fbc373592c2b78a0. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"In-Context (In-C)":"4","In-Memory (In-M)":"4"}. This record does not assert independent reproduction. ### LongLLaMA SST-2 Accuracy: 91.4. Result ID: 6a0c3826fbc373592c02e7ed. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"In-Context (InC)":"4","In-Memory (InM)":"18"}. This record does not assert independent reproduction. ### LongLLaMA SST-2 Accuracy: 91.4. Result ID: 6a2fa563fbc373592c2b78a3. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"In-Context (In-C)":"4","In-Memory (In-M)":"18"}. This record does not assert independent reproduction. ### MemLong SST-2 Accuracy: 91. Result ID: 6a0c3826fbc373592c02e7ee. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"In-Context (InC)":"4","In-Memory (InM)":"18"}. This record does not assert independent reproduction. ### MemLong SST-2 Accuracy: 91. Result ID: 6a2fa563fbc373592c2b78a4. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"In-Context (In-C)":"4","In-Memory (In-M)":"18"}. This record does not assert independent reproduction. ### OpenLLaMA w./ Rag SST-2 Accuracy: 90.9. Result ID: 6a0c3826fbc373592c02e7e9. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"In-Context (InC)":"4","In-Memory (InM)":"4"}. This record does not assert independent reproduction. ### OpenLLaMA w/ RAG SST-2 Accuracy: 90.9. Result ID: 6a2fa563fbc373592c2b789e. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"In-Context (In-C)":"4","In-Memory (In-M)":"4"}. This record does not assert independent reproduction. ### OpenLLaMA SST-2 Accuracy: 90.7. Result ID: 6a0c3826fbc373592c02e7e8. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"In-Context (InC)":"4","In-Memory (InM)":"N/A"}. This record does not assert independent reproduction. ### OpenLLaMA SST-2 Accuracy: 90.7. Result ID: 6a2fa563fbc373592c2b789d. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"In-Context (In-C)":"4","In-Memory (In-M)":"N/A"}. This record does not assert independent reproduction. ### LongLLaMA SST-2 Accuracy: 90.4. Result ID: 6a0c3826fbc373592c02e7ea. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"In-Context (InC)":"4","In-Memory (InM)":"4"}. This record does not assert independent reproduction. ### LongLLaMA SST-2 Accuracy: 90.4. Result ID: 6a2fa563fbc373592c2b789f. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"In-Context (In-C)":"4","In-Memory (In-M)":"4"}. This record does not assert independent reproduction. Page 1 of 1. This is an index page, not the full collection.