Paraphrase Attack Robustness on BookSum
98.49AUCSWEET
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| SWEETBase LLM=OPT-1.3B2025.12 | 98.49 | 92.5 | 97 | 74.92 | |
| EWDBase LLM=OPT-1.3B2025.12 | 98.21 | 94 | 97 | 72.77 | |
| KGWBase LLM=OPT-1.3B2025.12 | 97.77 | 90 | 93 | 71.95 | |
| DualGuardBase LLM=OPT-1.3B2025.12 | 97.6 | 92 | 95.5 | 96.56 | |
| XSIRBase LLM=OPT-1.3B2025.12 | 96.01 | 79 | 89 | 67.41 | |
| SIRBase LLM=OPT-1.3B2025.12 | 93.06 | 70.5 | 79.5 | 67.48 | |
| EWDBase LLM=Llama3.1-8B-Instruct2025.12 | 92.7 | 75 | 80.5 | 68.12 | |
| DualGuardBase LLM=Llama3.1-8B-Instruct2025.12 | 92.53 | 64.5 | 80.5 | 93.03 | |
| SWEETBase LLM=Llama3.1-8B-Instruct2025.12 | 90.4 | 65 | 76.5 | 66.54 | |
| KGWBase LLM=Llama3.1-8B-Instruct2025.12 | 89.99 | 68.5 | 73.5 | 69.2 | |
| XSIRBase LLM=Llama3.1-8B-Instruct2025.12 | 87.09 | 53 | 62.5 | 68.15 | |
| SIRBase LLM=Llama3.1-8B-Instruct2025.12 | 80.26 | 30.5 | 44 | 57.02 | |
| AARBase LLM=OPT-1.3B2025.12 | 78.34 | 10 | 28 | 65.71 | |
| SynthIDBase LLM=OPT-1.3B2025.12 | 75.59 | 19.5 | 32.5 | 60.29 | |
| AARBase LLM=Llama3.1-8B-Instruct2025.12 | 73.29 | 20.5 | 33.5 | 62.01 | |
| SynthIDBase LLM=Llama3.1-8B-Instruct2025.12 | 66.86 | 9 | 20 | 58.13 | |
| DIPmarkBase LLM=Llama3.1-8B-Instruct2025.12 | 55.12 | 9 | 17 | 52.59 | |
| DIPmarkBase LLM=OPT-1.3B2025.12 | 53.51 | 5.5 | 17.5 | 53.63 | |
| UnbiasedBase LLM=OPT-1.3B2025.12 | 51.62 | 5.3 | 10.6 | 49.45 | |
| UnbiasedBase LLM=Llama3.1-8B-Instruct2025.12 | 51.07 | 5.17 | 10.33 | 50.1 |