Text Classification on NKI Evaluation Suite (SST-2, SST-5, CoLA, TREC, CR, MPQA, Subj) (test)
95.7SST-2 AccuracyPGRA
Evaluation Results
| Method | Links | |||||||||
|---|---|---|---|---|---|---|---|---|---|---|
| PGRARetrieval=true, Backbone=T5-large (770M), d=82023.05 | 95.7 | 59.8 | 61.1 | 80.9 | 92.6 | 92.4 | 90.6 | 97.5 | 83.8 | |
| T5-largeRetrieval=false, Backbone=T5-large (770M)2023.05 | 95.2 | 59.2 | 60.7 | 80.8 | 92.1 | 91.5 | 90.7 | 97.3 | 83.4 | |
| RAGRetrieval=true, Backbone=T5-large (770M)2023.05 | 95.2 | 57.2 | 60.1 | 80.2 | 91.2 | 92.1 | 90.6 | 96.4 | 82.9 | |
| FiDRetrieval=true, Backbone=T5-large (770M)2023.05 | 94.8 | 59.5 | 60.2 | 80.8 | 92.4 | 92.5 | 90.6 | 97.5 | 83.5 | |
| PGRARetrieval=true, Backbone=T5-base (220M), d=162023.05 | 93.9 | 56.9 | 57 | 80.8 | 91.7 | 91.1 | 90.3 | 97 | 82.3 | |
| ICL (OPT-13b)Retrieval=false, Backbone=OPT-13b2023.05 | 93 | 46 | 1.8 | 26.8 | 73.2 | 61.7 | 71.6 | 51.1 | 53.2 | |
| RAGRetrieval=true, Backbone=T5-base (220M)2023.05 | 93 | 57.5 | 58.5 | 80.4 | 87.2 | 90.2 | 89.5 | 96.5 | 81.6 | |
| FiDRetrieval=true, Backbone=T5-base (220M)2023.05 | 92.2 | 56.6 | 56.9 | 80.8 | 91.3 | 90.1 | 89.8 | 96.6 | 81.8 | |
| T5-baseRetrieval=false, Backbone=T5-base (220M)2023.05 | 91.3 | 56.7 | 30.4 | 80.4 | 89.8 | 89.4 | 89.2 | 96 | 77.9 | |
| LM-BFFRetrieval=false, Backbone=T5-large (770M)2023.05 | 90.8 | 49 | 6.9 | 70.6 | 91.1 | 83.5 | 89.5 | 88.4 | 71.2 | |
| LM-BFFRetrieval=false, Backbone=T5-base (220M)2023.05 | 86 | 45.5 | 5.5 | 76.2 | 90 | 83.1 | 82.3 | 90.2 | 69.9 | |
| k-NNRetrieval=true, Backbone=T5-large (770M)2023.05 | 64.5 | 23.6 | 2.1 | 28.8 | 56.8 | 58.2 | 53.7 | 72.4 | 45 | |
| k-NNRetrieval=true, Backbone=T5-base (220M)2023.05 | 59.2 | 22.8 | 1 | 28 | 51.8 | 55.1 | 52.6 | 72.1 | 42.9 |