Watermarking Detection on BOOKSUM (test)
100Detection Rate (No Attack)KGW
Evaluation results
| Method | Links | |||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| KGWBackbone=OPT-1.3B2025.09 | 100 | 1.4 | 1.4 | 0.6 | 6 | 1.2 | 1.2 | 0.6 | 1.4 | 1.2 | 1.4 | |
| UPVBackbone=OPT-1.3B2025.09 | 100 | 90 | 89.2 | 73.6 | 98.6 | 100 | 99.8 | 99.4 | 100 | 100 | 99.4 | |
| SynthIDBackbone=OPT-1.3B2025.09 | 100 | 38.2 | 25.8 | 1.8 | 96 | 99.2 | 95.4 | 56 | 99.6 | 95.6 | 54.8 | |
| MorphMarkBackbone=OPT-1.3B2025.09 | 100 | 1.8 | 1.2 | 0.8 | 5 | 1.8 | 1.2 | 1.2 | 2 | 3.4 | 1 | |
| KGWBackbone=Mistral-7B2025.09 | 100 | 84.2 | 90.2 | 35.8 | 93 | 100 | 99.8 | 99 | 100 | 100 | 99.6 | |
| SIRBackbone=Mistral-7B2025.09 | 100 | 85 | 80.8 | 41.2 | 81.2 | 97.8 | 90 | 72.4 | 99.6 | 92 | 73 | |
| SynthIDBackbone=Mistral-7B2025.09 | 100 | 30.2 | 28.2 | 3 | 95.8 | 99.6 | 96.4 | 63.4 | 99.6 | 97.4 | 53.2 | |
| PMARK (Online)Backbone=Mistral-7B, Detection Mode=Online2025.09 | 100 | 94.4 | 96.6 | 96.8 | 94.4 | 99.6 | 98.2 | 82.9 | 100 | 99 | 93.6 | |
| PMARK (Online)Backbone=OPT-1.3B, Detection Mode=Online2025.09 | 99.8 | 97.4 | 96.8 | 95.4 | 93.2 | 99.8 | 96 | 86.2 | 99.8 | 98.2 | 90.8 | |
| EXPBackbone=Mistral-7B2025.09 | 99.8 | 36.4 | 47.2 | 10.4 | 93.6 | 98.4 | 95.6 | 76.8 | 98.6 | 97.2 | 82.2 | |
| MorphMarkBackbone=Mistral-7B2025.09 | 99.8 | 68.2 | 77.4 | 24.6 | 89 | 100 | 99.4 | 93.8 | 100 | 99 | 92.2 | |
| SIRBackbone=OPT-1.3B2025.09 | 99.6 | 84.2 | 80.6 | 49.8 | 77.4 | 96.2 | 86 | 65.2 | 98.6 | 92.2 | 69.8 | |
| KSEMSTAMPBackbone=OPT-1.3B2025.09 | 99.6 | 74.5 | 72.3 | 62.1 | 72.3 | 93.8 | 64.7 | 29.1 | 99.2 | 81.8 | 44.1 | |
| EXPGumbelBackbone=Mistral-7B2025.09 | 99.6 | 65.6 | 78.8 | 38.4 | 95.6 | 98.6 | 98.2 | 95.2 | 98.6 | 98.2 | 95 | |
| UPVBackbone=Mistral-7B2025.09 | 99.6 | 72.6 | 73 | 33.4 | 92.4 | 99.8 | 99.6 | 98.6 | 99.8 | 99.6 | 98.8 | |
| EXPGumbelBackbone=OPT-1.3B2025.09 | 99.4 | 12.4 | 12.4 | 13 | 21.4 | 23.8 | 20.4 | 13.4 | 24.8 | 19.8 | 16.4 | |
| PMARK (Offline)Backbone=OPT-1.3B, Detection Mode=Offline2025.09 | 99.4 | 93.2 | 95.2 | 94.2 | 96.4 | 98.6 | 98 | 86.7 | 99.4 | 98.4 | 87 | |
| PMARK (Offline)Backbone=Mistral-7B, Detection Mode=Offline2025.09 | 99.4 | 91.4 | 94.6 | 94.4 | 95.8 | 99.4 | 95.4 | 78.9 | 99.4 | 98.6 | 84.9 | |
| KSEMSTAMPBackbone=Mistral-7B2025.09 | 99 | 45.5 | 53.9 | 43.5 | 67.3 | 85.4 | 47.9 | 19.2 | 96.4 | 63.1 | 28.9 | |
| EXPBackbone=OPT-1.3B2025.09 | 98.8 | 4.2 | 5.4 | 4.4 | 5.4 | 3.8 | 2.2 | 1.6 | 6 | 2.4 | 3.2 | |
| SemStampBackbone=OPT-1.3B2025.09 | 97.7 | 89 | 90.7 | 79.6 | 87.8 | 97.6 | 89.4 | 67.5 | 97.7 | 93.2 | 78.1 | |
| SemStampBackbone=Mistral-7B2025.09 | 97.7 | 88.1 | 92.4 | 76 | 87.1 | 96.3 | 87.9 | 63.4 | 96.7 | 89.4 | 64.4 | |
| SimMarkBackbone=OPT-1.3B2025.09 | 88.2 | 23.4 | 29 | 22 | 22.2 | 68.4 | 39.8 | 11.8 | 80.4 | 45.8 | 19.6 | |
| SimMarkBackbone=Mistral-7B2025.09 | 78 | 19.8 | 28.8 | 23.4 | 21 | 54.6 | 27.6 | 16.4 | 70.6 | 40.4 | 17.4 |