Text Classification on SST-2 (test)
98AccuracyUpper Bound
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| Upper BoundModel=Claude, Evaluation Protocol=non-private baseline, epsilon=infinity2023.05 | 98 | — | — | |
| T5-11BType=SOTA, Training=Full fine-tuning2023.11 | 97.5 | — | — | |
| EFLTraining setup=Full training dataset, Backbone=RoBERTa-large2021.04 | 96.9 | — | — | |
| Fine-tuningTraining setup=Full training dataset, Backbone=RoBERTa-large2021.04 | 96.4 | — | — | |
| PRL2025.12 | 96.32 | — | — | |
| Ens. Acc.Model=Claude, Evaluation Protocol=ensemble, epsilon=infinity2023.05 | 96 | — | — | |
| PIAST (E)variant=Ensemble2025.12 | 95.88 | — | — | |
| NI2025.12 | 95.77 | — | — | |
| PromptPATEModel=Claude, epsilon=0.0482023.05 | 95.7 | — | — | |
| PIAST (LOO)variant=Leave-one-out2025.12 | 95.7 | — | — | |
| SPINBackbone=Flan-T5-XL, Variant=SPIN2023.11 | 95.64 | — | 12.85 | |
| PIAST2025.12 | 95.35 | — | — | |
| Non-private One-shot BaselineBackbone=GPT3-Curie, Protocol=One-shot, Shots=12023.05 | 95.2 | — | — | |
| PIAST (I)variant=Iterative2025.12 | 95.04 | — | — | |
| PRO-T5 HUBERK=3, delta=3, Attack Method=TextFooler, Num Perturbed Words=02024.10 | 95 | — | — | |
| GA2025.12 | 94.65 | — | — | |
| SPINBackbone=RoBERTa2023.11 | 94.38 | — | — | |
| RoBERTaEvaluation Variant=Base2023.11 | 94.03 | — | — | |
| Private EnsembleBackbone=GPT3-Curie, Protocol=Ensemble2023.05 | 94 | — | — | |
| Non-private One-shot BaselineBackbone=GPT3-Babbage, Protocol=One-shot, Shots=12023.05 | 93.8 | — | — | |
| APO2025.12 | 93.71 | — | — | |
| DE2025.12 | 93.29 | — | — | |
| SPINBackbone=GPT2-XL, Variant=SPIN2023.11 | 93.23 | — | 3.57 | |
| Lower BoundModel=Claude, Evaluation Protocol=zero-shot, epsilon=02023.05 | 92.7 | — | — | |
| PromptPATEBackbone=GPT3-Curie, Transfer Setting=OOD, Public Dataset=imdb, Shots=12023.05 | 92.7 | 0.154 | — | |
| MI2025.12 | 92.7 | — | — | |
| TinyBERT2021.06 | 92.6 | — | — | |
| SPINBackbone=Flan-T5, Variant=SPIN2023.11 | 92.32 | — | 17.97 | |
| SPINBackbone=GPT22023.11 | 92.32 | — | — | |
| Fine-tuned BERTSNN=false, Spike=false, PE=false, Param (M)=109.82024.05 | 92.31 | — | — | |
| HDGCN2021.06 | 92.3 | — | — | |
| PromptPATEBackbone=GPT3-Curie, Transfer Setting=IID, Public Dataset=sst2, Shots=12023.05 | 92.3 | 0.147 | — | |
| PINFew-shot=5-shot, Policy LM=OPT-125M2024.07 | 92 | — | — | |
| bmLSTM2017.08 | 91.8 | — | — | |
| Byte mLSTMArchitecture=Byte mLSTM2018.05 | 91.8 | — | — | |
| GPT2Evaluation Variant=Base2023.11 | 91.51 | — | — | |
| RLPromptFew-shot=5-shot, Policy LM=OPT-125M2024.07 | 91.5 | — | — | |
| APE2025.12 | 91.23 | — | — | |
| BCN with SuBiLSTM-Tied+CoVeArchitecture=BCN, Encoder=SuBiLSTM-Tied, Embeddings=CoVe2018.05 | 91.2 | — | — | |
| SPINBackbone=DistilBERT2023.11 | 91.19 | — | — | |
| DistilBERTEvaluation Variant=Base2023.11 | 91.05 | — | — | |
| BCN with SuBiLSTM+CoVeArchitecture=BCN, Encoder=SuBiLSTM, Embeddings=CoVe2018.05 | 91 | — | — | |
| HDGCN-static2021.06 | 90.8 | — | — | |
| EFLTraining setup=Few-shot, K (samples per class)=8, Backbone=RoBERTa-large2021.04 | 90.8 | — | — | |
| SPINBackbone=RoBERTa, Variant=SPIN2023.11 | 90.59 | — | 7.77 | |
| BCN with 2-layer BiLSTM+CoVeArchitecture=BCN, Encoder=2-layer BiLSTM, Embeddings=CoVe2018.05 | 90.5 | — | — | |
| BCN+Char+CoVeembeddings=GloVe, character n-gram, CoVe2017.08 | 90.3 | — | — | |
| BCN+Char+CoVeArchitecture=BCN, Embeddings=Char, CoVe2018.05 | 90.3 | — | — | |
| SPINBackbone=GPT2-M, Variant=SPIN2023.11 | 90.25 | — | 4.8 | |
| BCN with BiLSTM+CoVeArchitecture=BCN, Encoder=BiLSTM, Embeddings=CoVe2018.05 | 90.1 | — | — | |
| PRO-T5 l1K=3, Attack Method=TextFooler, Num Perturbed Words=02024.10 | 90.1 | — | — | |
| GPT2-XLBackbone=GPT2-XL, Variant=Base2023.11 | 90.02 | — | — | |
| Private EnsembleBackbone=GPT3-Babbage, Protocol=Ensemble2023.05 | 90 | — | — | |
| BCN with SuBiLSTMArchitecture=BCN, Encoder=SuBiLSTM2018.05 | 89.8 | — | — | |
| NSE2017.08 | 89.7 | — | — | |
| NSEArchitecture=NSE2018.05 | 89.7 | — | — | |
| BCN with SuBiLSTM-TiedArchitecture=BCN, Encoder=SuBiLSTM-Tied2018.05 | 89.7 | — | — | |
| TE-LSTM2017.08 | 89.6 | — | — | |
| BCN with 2-layer BiLSTMArchitecture=BCN, Encoder=2-layer BiLSTM2018.05 | 89.5 | — | — | |
| CT-LSTM2017.08 | 89.4 | — | — | |
| CNN-NSU2021.06 | 89.4 | — | — | |
| BCN with BiLSTMArchitecture=BCN, Encoder=BiLSTM2018.05 | 89.3 | — | — | |
| BERT2021.06 | 89.3 | — | — | |
| P-LSTM2017.08 | 89.2 | — | — | |
| T5Attack Method=TextFooler, Num Perturbed Words=02024.10 | 89.1 | — | — | |
| T5Attack Method=TextFooler, Num Perturbed Words=12024.10 | 89.1 | — | — | |
| T5Attack Method=TextFooler, Num Perturbed Words=22024.10 | 89.1 | — | — | |
| PRO-T5 HUBER-MCPK=3, delta=9, gamma=15, Attack Method=TextFooler, Num Perturbed Words=02024.10 | 89.1 | — | — | |
| InstructionsFew-shot=5-shot2024.07 | 89 | — | — | |
| SPINBackbone=Flan-T5-S, Variant=SPIN2023.11 | 88.99 | — | 15.32 | |
| PromptPATEBackbone=GPT3-Babbage, Transfer Setting=IID, Public Dataset=sst2, Shots=12023.05 | 88.8 | 0.178 | — | |
| LK-MTL2021.06 | 88.5 | — | — | |
| Paragraph-Vec2018.06 | 87.8 | — | — | |
| SPINBackbone=GPT2, Variant=SPIN2023.11 | 87.73 | — | 2.14 | |
| Standard DR-AGG2018.06 | 87.6 | — | — | |
| CNN-non-static2018.06 | 87.2 | — | — | |
| MT-LSTM (F2S)2018.06 | 87.2 | — | — | |
| Reverse DR-AGG2018.06 | 87.2 | — | — | |
| PromptPATEBackbone=GPT3-Babbage, Transfer Setting=OOD, Public Dataset=imdb, Shots=12023.05 | 87.2 | 0.187 | — | |
| Our method (Weight Net)Imbalance Factor=1000:202022.08 | 87.14 | — | — | |
| Our method (Weight Net)Imbalance Factor=1000:502022.08 | 87.13 | — | — | |
| Our method (Weight Net)Imbalance Factor=1000:102022.08 | 87.1 | — | — | |
| GrIPSFew-shot=5-shot2024.07 | 87.1 | — | — | |
| PRO-T5 MCPK=4, gamma=3, Attack Method=TextFooler, Num Perturbed Words=02024.10 | 87.1 | — | — | |
| Our method (Weight Net)Imbalance Factor=1000:1002022.08 | 87.08 | — | — | |
| Max pooling2018.06 | 87 | — | — | |
| CapNets2021.06 | 86.8 | — | — | |
| Logit AdjustmentImbalance Factor=1000:502022.08 | 86.61 | — | — | |
| Logit AdjustmentImbalance Factor=1000:202022.08 | 86.51 | — | — | |
| Logit AdjustmentImbalance Factor=1000:102022.08 | 86.5 | — | — | |
| Self-attention2018.06 | 86.4 | — | — | |
| DRNN2021.06 | 86.4 | — | — | |
| Logit AdjustmentImbalance Factor=1000:1002022.08 | 86.37 | — | — | |
| GPT2-MBackbone=GPT2-M, Variant=Base2023.11 | 86.12 | — | — | |
| In-Context DemonstrationFew-shot=5-shot2024.07 | 85.9 | — | — | |
| GPT2Backbone=GPT2, Variant=Base2023.11 | 85.89 | — | — | |
| Stilts-CloseTraining setup=Few-shot, K (samples per class)=8, Backbone=RoBERTa-large2021.04 | 85.5 | — | — | |
| Average pooling2018.06 | 85.2 | — | — | |
| Flan-T5-XLBackbone=Flan-T5-XL, Variant=Base2023.11 | 84.75 | — | — | |
| RoBERTaBackbone=RoBERTa, Variant=Base2023.11 | 84.06 | — | — |