Adversarial Text Detection on AG News
96.7F1 ScoreVoteTRANS_same
Evaluation results
| Method | Links | ||
|---|---|---|---|
| VoteTRANS_sameTarget Model=DistilBERT, Attack=TextFooler, Auxiliary Attack=TextFooler2023.06 | 96.7 | 98.4 | |
| VoteTRANS_sameTarget Model=CNN, Attack=PWWS, Auxiliary Attack=PWWS2023.06 | 96.5 | 99.2 | |
| VoteTRANS_sameTarget Model=DistilBERT, Attack=IGA, Auxiliary Attack=IGA2023.06 | 96.5 | 99.2 | |
| WDRTarget Model=DistilBERT, Attack=TextFooler, Detector training configuration=(DistilBERT, AG News, PWWS)2023.06 | 95.7 | 97.3 | |
| VoteTRANS_diffTarget Model=DistilBERT, Attack=TextFooler, Auxiliary Attack=PWWS2023.06 | 95.7 | 98 | |
| VoteTRANS_sameTarget Model=LSTM, Attack=PWWS, Auxiliary Attack=PWWS2023.06 | 95.5 | 98.4 | |
| VoteTRANS_sameTarget Model=DistilBERT, Attack=PWWS, Auxiliary Attack=PWWS2023.06 | 95.3 | 97.6 | |
| VoteTRANS_sameTarget Model=BERT, Attack=PWWS, Auxiliary Attack=PWWS2023.06 | 94.6 | 98 | |
| WDRTarget Model=LSTM, Attack=PWWS, Detector training configuration=(DistilBERT, AG News, PWWS)2023.06 | 94 | 94.2 | |
| WDRTarget Model=DistilBERT, Attack=PWWS, Detector training configuration=(DistilBERT, AG News, PWWS)2023.06 | 93.6 | 94.8 | |
| VoteTRANS_diffTarget Model=DistilBERT, Attack=IGA, Auxiliary Attack=PWWS2023.06 | 93.2 | 95.6 | |
| WDRTarget Model=BERT, Attack=PWWS, Detector training configuration=(DistilBERT, AG News, PWWS)2023.06 | 92.5 | 93 | |
| WDRTarget Model=CNN, Attack=PWWS, Detector training configuration=(DistilBERT, AG News, PWWS)2023.06 | 91.1 | 91.2 | |
| FGWSTarget Model=CNN, Attack=PWWS, Detector training configuration=(DistilBERT, AG News, PWWS)2023.06 | 90.6 | 87.6 | |
| FGWSTarget Model=DistilBERT, Attack=PWWS, Detector training configuration=(DistilBERT, AG News, PWWS)2023.06 | 89.5 | 84.6 | |
| FGWSTarget Model=LSTM, Attack=PWWS, Detector training configuration=(DistilBERT, AG News, PWWS)2023.06 | 88.9 | 84.9 | |
| FGWSTarget Model=BERT, Attack=PWWS, Detector training configuration=(DistilBERT, AG News, PWWS)2023.06 | 88.7 | 83.2 | |
| FGWSTarget Model=DistilBERT, Attack=TextFooler, Detector training configuration=(DistilBERT, AG News, PWWS)2023.06 | 87 | 79.4 | |
| WDRTarget Model=DistilBERT, Attack=IGA, Detector training configuration=(DistilBERT, AG News, PWWS)2023.06 | 86.7 | 93.6 | |
| WDRTarget Model=DistilBERT, Attack=BAE, Detector training configuration=(DistilBERT, AG News, PWWS)2023.06 | 86.4 | 94.5 | |
| VoteTRANS_sameTarget Model=DistilBERT, Attack=BAE, Auxiliary Attack=BAE2023.06 | 85.8 | 93.2 | |
| VoteTRANS_diffTarget Model=DistilBERT, Attack=BAE, Auxiliary Attack=PWWS2023.06 | 85.3 | 92.8 | |
| FGWSTarget Model=DistilBERT, Attack=IGA, Detector training configuration=(DistilBERT, AG News, PWWS)2023.06 | 68.6 | 58.3 | |
| FGWSTarget Model=DistilBERT, Attack=BAE, Detector training configuration=(DistilBERT, AG News, PWWS)2023.06 | 55.8 | 44 |