Watermark Detection on C4 (Specific FPR Thresholds)
1TPR @ FPR=1%KGW
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| KGWBackbone=OPT-1.3B, Paraphrasing Attack=No Attack2026.05 | 1 | 1 | 1 | |
| SynthIDBackbone=OPT-1.3B, Paraphrasing Attack=No Attack2026.05 | 1 | 1 | 1 | |
| SIRBackbone=OPT-1.3B, Attack=No Attack2025.09 | 0.998 | 1 | 0.999 | |
| EXPBackbone=OPT-1.3B, Paraphrasing Attack=No Attack2026.05 | 0.996 | 0.996 | 0.998 | |
| AliMarkBackbone=Qwen3-1.7B, Attack=No Attack2026.05 | 0.992 | 0.996 | 0.999 | |
| AliMarkBackbone=OPT-1.3B, Paraphrasing Attack=No Attack2026.05 | 0.992 | 0.994 | 0.999 | |
| KGWBackbone=Qwen3-1.7B, Attack=No Attack2026.05 | 0.984 | 0.992 | 1 | |
| SynthIDBackbone=Qwen3-1.7B, Attack=No Attack2026.05 | 0.98 | 0.982 | 0.999 | |
| EXPBackbone=Qwen3-1.7B, Attack=No Attack2026.05 | 0.976 | 0.98 | 0.994 | |
| PMarkBackbone=Qwen3-1.7B, Attack=No Attack2026.05 | 0.974 | 0.974 | 0.995 | |
| SIRBackbone=OPT-1.3B, Paraphrasing Attack=No Attack2026.05 | 0.954 | 0.982 | 0.999 | |
| PMARK (Online)Backbone=Mistral-7B, Attack=Doc-P (GPT)2025.09 | 0.952 | 0.988 | 0.997 | |
| PMarkBackbone=OPT-1.3B, Paraphrasing Attack=No Attack2026.05 | 0.948 | 0.95 | 0.989 | |
| SemStampBackbone=Mistral-7B, Attack=No Attack2025.09 | 0.925 | 0.958 | 0.983 | |
| SemStampBackbone=Qwen3-1.7B, Attack=No Attack2026.05 | 0.895 | 0.919 | 0.993 | |
| PMarkBackbone=OPT-1.3B, Paraphrasing Attack=Parrot2026.05 | 0.886 | 0.894 | 0.981 | |
| SemStampBackbone=OPT-1.3B, Paraphrasing Attack=No Attack2026.05 | 0.842 | 0.849 | 0.978 | |
| AliMarkBackbone=OPT-1.3B, Paraphrasing Attack=Parrot2026.05 | 0.836 | 0.912 | 0.989 | |
| PMarkBackbone=Qwen3-1.7B, Attack=Parrot2026.05 | 0.83 | 0.844 | 0.981 | |
| PMarkBackbone=Qwen3-1.7B, Attack=Pegasus2026.05 | 0.822 | 0.842 | 0.976 | |
| PMarkBackbone=OPT-1.3B, Paraphrasing Attack=Pegasus2026.05 | 0.786 | 0.796 | 0.961 | |
| AliMarkBackbone=Qwen3-1.7B, Attack=Parrot2026.05 | 0.784 | 0.878 | 0.99 | |
| SIRBackbone=Qwen3-1.7B, Attack=No Attack2026.05 | 0.756 | 0.924 | 0.996 | |
| SIRBackbone=OPT-1.3B, Attack=Doc-P (GPT)2025.09 | 0.742 | 0.886 | 0.977 | |
| AliMarkBackbone=Qwen3-1.7B, Attack=Pegasus2026.05 | 0.732 | 0.846 | 0.987 | |
| AliMarkBackbone=OPT-1.3B, Paraphrasing Attack=Pegasus2026.05 | 0.696 | 0.812 | 0.982 | |
| k-SemStampBackbone=Qwen3-1.7B, Attack=No Attack2026.05 | 0.565 | 0.756 | 0.996 | |
| SemStampBackbone=Qwen3-1.7B, Attack=Parrot2026.05 | 0.534 | 0.589 | 0.961 | |
| SemStampBackbone=OPT-1.3B, Paraphrasing Attack=Parrot2026.05 | 0.534 | 0.569 | 0.93 | |
| EXPBackbone=OPT-1.3B, Paraphrasing Attack=Parrot2026.05 | 0.511 | 0.543 | 0.892 | |
| k-SemStampBackbone=Mistral-7B, Attack=Doc-P (Pegasus)2025.09 | 0.492 | 0.684 | 0.932 | |
| SemStampBackbone=Qwen3-1.7B, Attack=Pegasus2026.05 | 0.485 | 0.53 | 0.941 | |
| EXPBackbone=OPT-1.3B, Paraphrasing Attack=Pegasus2026.05 | 0.475 | 0.519 | 0.882 | |
| SemStampBackbone=OPT-1.3B, Paraphrasing Attack=Pegasus2026.05 | 0.471 | 0.535 | 0.927 | |
| SynthIDBackbone=OPT-1.3B, Paraphrasing Attack=Parrot2026.05 | 0.436 | 0.514 | 0.916 | |
| AliMarkBackbone=Qwen3-1.7B, Attack=DIPPER2026.05 | 0.414 | 0.562 | 0.925 | |
| PMarkBackbone=Qwen3-1.7B, Attack=DIPPER2026.05 | 0.384 | 0.396 | 0.852 | |
| AliMarkBackbone=OPT-1.3B, Paraphrasing Attack=DIPPER2026.05 | 0.37 | 0.498 | 0.915 | |
| SynthIDBackbone=OPT-1.3B, Paraphrasing Attack=Pegasus2026.05 | 0.368 | 0.466 | 0.927 | |
| AliMarkBackbone=OPT-1.3B, Paraphrasing Attack=GPT-3.52026.05 | 0.36 | 0.516 | 0.923 | |
| AliMarkBackbone=Qwen3-1.7B, Attack=GPT-3.52026.05 | 0.292 | 0.444 | 0.904 | |
| PMarkBackbone=OPT-1.3B, Paraphrasing Attack=DIPPER2026.05 | 0.286 | 0.296 | 0.842 | |
| PMarkBackbone=OPT-1.3B, Paraphrasing Attack=GPT-3.52026.05 | 0.264 | 0.282 | 0.851 | |
| PMarkBackbone=Qwen3-1.7B, Attack=GPT-3.52026.05 | 0.252 | 0.268 | 0.819 | |
| SIRBackbone=Qwen3-1.7B, Attack=Parrot2026.05 | 0.24 | 0.644 | 0.97 | |
| k-SemStampBackbone=OPT-1.3B, Paraphrasing Attack=No Attack2026.05 | 0.233 | 0.911 | 0.993 | |
| SynthIDBackbone=Qwen3-1.7B, Attack=Parrot2026.05 | 0.232 | 0.326 | 0.876 | |
| EXPBackbone=OPT-1.3B, Paraphrasing Attack=DIPPER2026.05 | 0.21 | 0.248 | 0.767 | |
| k-SemStampBackbone=Qwen3-1.7B, Attack=Parrot2026.05 | 0.174 | 0.257 | 0.952 | |
| SemStampBackbone=Qwen3-1.7B, Attack=DIPPER2026.05 | 0.162 | 0.199 | 0.796 | |
| SynthIDBackbone=Qwen3-1.7B, Attack=Pegasus2026.05 | 0.154 | 0.276 | 0.834 | |
| SemStampBackbone=OPT-1.3B, Paraphrasing Attack=DIPPER2026.05 | 0.147 | 0.179 | 0.791 | |
| SimMarkBackbone=OPT-1.3B, Paraphrasing Attack=No Attack2026.05 | 0.138 | 0.754 | 0.986 | |
| SemStampBackbone=OPT-1.3B, Paraphrasing Attack=GPT-3.52026.05 | 0.127 | 0.168 | 0.752 | |
| SemStampBackbone=Qwen3-1.7B, Attack=GPT-3.52026.05 | 0.124 | 0.145 | 0.77 | |
| k-SemStampBackbone=Qwen3-1.7B, Attack=Pegasus2026.05 | 0.122 | 0.174 | 0.936 | |
| SIRBackbone=Qwen3-1.7B, Attack=Pegasus2026.05 | 0.086 | 0.388 | 0.927 | |
| SimMarkBackbone=OPT-1.3B, Paraphrasing Attack=Pegasus2026.05 | 0.066 | 0.582 | 0.974 | |
| EXPBackbone=OPT-1.3B, Paraphrasing Attack=GPT-3.52026.05 | 0.062 | 0.082 | 0.646 | |
| SynthIDBackbone=OPT-1.3B, Paraphrasing Attack=DIPPER2026.05 | 0.058 | — | 0.709 | |
| SimMarkBackbone=OPT-1.3B, Paraphrasing Attack=Parrot2026.05 | 0.04 | 0.606 | 0.977 | |
| SynthIDBackbone=OPT-1.3B, Paraphrasing Attack=GPT-3.52026.05 | 0.034 | 0.054 | 0.661 | |
| SynthIDBackbone=Qwen3-1.7B, Attack=GPT-3.52026.05 | 0.032 | 0.054 | 0.624 | |
| SimMarkBackbone=Qwen3-1.7B, Attack=No Attack2026.05 | 0.032 | 0.622 | 0.964 | |
| k-SemStampBackbone=OPT-1.3B, Paraphrasing Attack=Parrot2026.05 | 0.03 | 0.399 | 0.948 | |
| k-SemStampBackbone=OPT-1.3B, Paraphrasing Attack=Pegasus2026.05 | 0.024 | 0.339 | 0.935 | |
| SIRBackbone=Qwen3-1.7B, Attack=GPT-3.52026.05 | 0.022 | 0.136 | 0.823 | |
| EXPBackbone=Qwen3-1.7B, Attack=Pegasus2026.05 | 0.02 | 0.02 | 0.513 | |
| k-SemStampBackbone=Qwen3-1.7B, Attack=GPT-3.52026.05 | 0.019 | 0.046 | 0.783 | |
| SimMarkBackbone=Qwen3-1.7B, Attack=Pegasus2026.05 | 0.018 | 0.48 | 0.945 | |
| SynthIDBackbone=Qwen3-1.7B, Attack=DIPPER2026.05 | 0.016 | 0.034 | 0.645 | |
| SimMarkBackbone=OPT-1.3B, Paraphrasing Attack=DIPPER2026.05 | 0.012 | 0.202 | 0.891 | |
| k-SemStampBackbone=Qwen3-1.7B, Attack=DIPPER2026.05 | 0.011 | 0.027 | 0.787 | |
| EXPBackbone=Qwen3-1.7B, Attack=Parrot2026.05 | 0.01 | 0.014 | 0.51 | |
| KGWBackbone=OPT-1.3B, Paraphrasing Attack=Pegasus2026.05 | 0.01 | 0.01 | 0.5 | |
| SimMarkBackbone=Qwen3-1.7B, Attack=Parrot2026.05 | 0.008 | 0.462 | 0.948 | |
| KGWBackbone=Qwen3-1.7B, Attack=Pegasus2026.05 | 0.006 | 0.01 | 0.421 | |
| KGWBackbone=Qwen3-1.7B, Attack=DIPPER2026.05 | 0.006 | 0.012 | 0.407 | |
| EXPBackbone=Qwen3-1.7B, Attack=DIPPER2026.05 | 0.006 | 0.018 | 0.505 | |
| SIRBackbone=Qwen3-1.7B, Attack=DIPPER2026.05 | 0.006 | 0.076 | 0.793 | |
| KGWBackbone=OPT-1.3B, Paraphrasing Attack=DIPPER2026.05 | 0.006 | 0.006 | 0.535 | |
| SimMarkBackbone=OPT-1.3B, Paraphrasing Attack=GPT-3.52026.05 | 0.006 | 0.086 | 0.808 | |
| k-SemStampBackbone=OPT-1.3B, Paraphrasing Attack=GPT-3.52026.05 | 0.003 | 0.061 | 0.776 | |
| SimMarkBackbone=Qwen3-1.7B, Attack=DIPPER2026.05 | 0.002 | 0.148 | 0.865 | |
| KGWBackbone=OPT-1.3B, Paraphrasing Attack=GPT-3.52026.05 | 0.002 | 0.004 | 0.465 | |
| SIRBackbone=OPT-1.3B, Paraphrasing Attack=Pegasus2026.05 | 0.002 | 0.012 | 0.456 | |
| k-SemStampBackbone=OPT-1.3B, Paraphrasing Attack=DIPPER2026.05 | 0.002 | 0.09 | 0.815 | |
| KGWBackbone=Qwen3-1.7B, Attack=Parrot2026.05 | 0 | 0.008 | 0.456 | |
| KGWBackbone=Qwen3-1.7B, Attack=GPT-3.52026.05 | 0 | 0.01 | 0.466 | |
| EXPBackbone=Qwen3-1.7B, Attack=GPT-3.52026.05 | 0 | 0 | 0.504 | |
| SimMarkBackbone=Qwen3-1.7B, Attack=GPT-3.52026.05 | 0 | 0.048 | 0.728 | |
| KGWBackbone=OPT-1.3B, Paraphrasing Attack=Parrot2026.05 | 0 | 0.002 | 0.541 | |
| SIRBackbone=OPT-1.3B, Paraphrasing Attack=Parrot2026.05 | 0 | 0.002 | 0.376 | |
| SIRBackbone=OPT-1.3B, Paraphrasing Attack=DIPPER2026.05 | 0 | 0.002 | 0.422 | |
| SIRBackbone=OPT-1.3B, Paraphrasing Attack=GPT-3.52026.05 | 0 | 0.004 | 0.449 |