Text Classification on NLU Tasks (SST-2, MR, Subj, SST-5, MPQA)
95.4SST-2 AccuracyKGERMAR
Evaluation results
| Method | Links | ||||||
|---|---|---|---|---|---|---|---|
| KGERMARIn-Context (In-C)=4, In-Memory (In-M)=42026.06 | 95.4 | 91.4 | 69.9 | 47.4 | 88.9 | 78.6 | |
| KGERMARIn-Context (In-C)=4, In-Memory (In-M)=182026.06 | 95.4 | 91.4 | 69.9 | 47.4 | 88.9 | 78.6 | |
| ERMARIn-Context (InC)=20, In-Memory (InM)=182025.03 | 94.7 | 91.7 | 82.8 | 47 | 86.5 | 80.54 | |
| ERMARIn-Context (In-C)=20, In-Memory (In-M)=182026.06 | 94.7 | 91.7 | 82.8 | 47 | 86.5 | 80.54 | |
| KGERMARIn-Context (In-C)=20, In-Memory (In-M)=182026.06 | 94.3 | 91.3 | 85.4 | 46.9 | 88.1 | 81.2 | |
| LongLLaMAIn-Context (InC)=20, In-Memory (InM)=182025.03 | 94.1 | 90.8 | 64.2 | 41.4 | 72.1 | 72.7 | |
| LongLLaMAIn-Context (In-C)=20, In-Memory (In-M)=182026.06 | 94.1 | 90.8 | 64.2 | 41.4 | 72.1 | 72.7 | |
| ERMARIn-Context (InC)=4, In-Memory (InM)=42025.03 | 93.6 | 90.8 | 65.3 | 45.8 | 85.2 | 76.14 | |
| ERMARIn-Context (InC)=4, In-Memory (InM)=182025.03 | 93.6 | 90.8 | 65.3 | 45.9 | 85.2 | 76.16 | |
| OpenLLaMAIn-Context (InC)=20, In-Memory (InM)=N/A2025.03 | 93.6 | 91.2 | 55.4 | 38.2 | 66.4 | 69 | |
| ERMARIn-Context (In-C)=4, In-Memory (In-M)=42026.06 | 93.6 | 90.8 | 65.3 | 45.8 | 85.2 | 76.14 | |
| ERMARIn-Context (In-C)=4, In-Memory (In-M)=182026.06 | 93.6 | 90.8 | 65.3 | 45.9 | 85.2 | 76.16 | |
| OpenLLaMAIn-Context (In-C)=20, In-Memory (In-M)=N/A2026.06 | 93.6 | 91.2 | 55.4 | 38.2 | 66.4 | 69 | |
| MemLongIn-Context (InC)=20, In-Memory (InM)=182025.03 | 93.5 | 93.8 | 65.8 | 43.3 | 70.6 | 73.4 | |
| MemLongIn-Context (In-C)=20, In-Memory (In-M)=182026.06 | 93.5 | 93.8 | 65.8 | 43.3 | 70.6 | 73.4 | |
| OpenLLaMA w./ RagIn-Context (InC)=20, In-Memory (InM)=182025.03 | 92.2 | 91.3 | 75.8 | 39.8 | 57.6 | 71.3 | |
| OpenLLaMA w/ RAGIn-Context (In-C)=20, In-Memory (In-M)=182026.06 | 92.2 | 91.3 | 75.8 | 39.8 | 57.6 | 71.3 | |
| MemLongIn-Context (InC)=4, In-Memory (InM)=42025.03 | 91.5 | 84.5 | 61.5 | 41.4 | 70.2 | 69.8 | |
| MemLongIn-Context (In-C)=4, In-Memory (In-M)=42026.06 | 91.5 | 84.5 | 61.5 | 41.4 | 70.2 | 69.8 | |
| LongLLaMAIn-Context (InC)=4, In-Memory (InM)=182025.03 | 91.4 | 87.1 | 59.1 | 41 | 64.5 | 68.7 | |
| LongLLaMAIn-Context (In-C)=4, In-Memory (In-M)=182026.06 | 91.4 | 87.1 | 59.1 | 41 | 64.5 | 68.7 | |
| MemLongIn-Context (InC)=4, In-Memory (InM)=182025.03 | 91 | 89.6 | 61.7 | 43.5 | 69.4 | 71 | |
| MemLongIn-Context (In-C)=4, In-Memory (In-M)=182026.06 | 91 | 89.6 | 61.7 | 43.5 | 69.4 | 71 | |
| OpenLLaMA w./ RagIn-Context (InC)=4, In-Memory (InM)=42025.03 | 90.9 | 90.5 | 61.6 | 39.2 | 63.2 | 69.1 | |
| OpenLLaMA w/ RAGIn-Context (In-C)=4, In-Memory (In-M)=42026.06 | 90.9 | 90.5 | 61.6 | 39.2 | 63.2 | 69.1 | |
| OpenLLaMAIn-Context (InC)=4, In-Memory (InM)=N/A2025.03 | 90.7 | 84 | 58.2 | 41 | 70.5 | 68.9 | |
| OpenLLaMAIn-Context (In-C)=4, In-Memory (In-M)=N/A2026.06 | 90.7 | 84 | 58.2 | 41 | 70.5 | 68.9 | |
| LongLLaMAIn-Context (InC)=4, In-Memory (InM)=42025.03 | 90.4 | 83.9 | 64.3 | 40 | 64.2 | 68.6 | |
| LongLLaMAIn-Context (In-C)=4, In-Memory (In-M)=42026.06 | 90.4 | 83.9 | 64.3 | 40 | 64.2 | 68.6 |