Large Language Model Watermarking on BOOKSUM (test)
4.38Average RankSynthID
Evaluation results
| Method | Links | |||||||||
|---|---|---|---|---|---|---|---|---|---|---|
| SynthIDBackbone=Nemotron-Nano-9B-v2-Base2026.05 | 4.38 | 99.8 | 31.4 | 15.6 | 21.2 | 47.8 | 33.4 | 18.8 | -0.67 | |
| PMarkBackbone=Mistral-Small-3.1-24B-Base-25032026.05 | 4.61 | 98 | 51.5 | 57.5 | 77 | 57.2 | 37.6 | 5.2 | -1.15 | |
| k-SemStampBackbone=Mistral-Small-3.1-24B-Base-25032026.05 | 4.94 | 98.6 | 61.7 | 43.1 | 65.9 | 51.4 | 47.2 | 1.4 | -0.82 | |
| SemStampBackbone=Mistral-Small-3.1-24B-Base-25032026.05 | 5.04 | 96 | 49.8 | 39.8 | 58.6 | 53.2 | 39.2 | 7.6 | -0.72 | |
| NoneBackbone=Nemotron-Nano-9B-v2-Base2026.05 | 5.06 | — | — | — | — | — | — | — | 0 | |
| SynthIDBackbone=Mistral-Small-3.1-24B-Base-25032026.05 | 5.11 | 99.8 | 23.8 | 10.4 | 12.6 | 54.2 | 38.6 | 7.2 | -0.65 | |
| SAMARKBackbone=Mistral-Small-3.1-24B-Base-2503, Optimization Status=Full Pipeline2026.05 | 5.14 | 98 | 90.5 | 88.1 | 87.1 | 47.9 | 45.2 | 6.9 | -0.62 | |
| SemStampBackbone=Nemotron-Nano-9B-v2-Base2026.05 | 5.24 | 97 | 46.9 | 34.5 | 64.3 | 39.2 | 41.6 | 19.2 | 0.18 | |
| UPVBackbone=Nemotron-Nano-9B-v2-Base2026.05 | 5.42 | 99.4 | 71 | 48.8 | 61.6 | 39.2 | 38.2 | 22.6 | 0.36 | |
| EXPBackbone=Mistral-Small-3.1-24B-Base-25032026.05 | 5.56 | 99.8 | 38.2 | 17.2 | 29.6 | 52.4 | 46.2 | 1.4 | -0.2 | |
| MorphMarkBackbone=Nemotron-Nano-9B-v2-Base2026.05 | 5.57 | 100 | 74 | 46.2 | 64 | 36.4 | 40.4 | 23.2 | 0.51 | |
| k-SemStampBackbone=Nemotron-Nano-9B-v2-Base2026.05 | 5.58 | 97.6 | 51.7 | 37.1 | 63.7 | 35 | 45 | 20 | 0.52 | |
| SAMARKBackbone=Nemotron-Nano-9B-v2-Base, Optimization Status=Full Pipeline2026.05 | 5.75 | 98.8 | 92.6 | 90.2 | 88.2 | 34.2 | 46 | 19.8 | 0.69 | |
| NoneBackbone=Mistral-Small-3.1-24B-Base-25032026.05 | 5.76 | — | — | — | — | — | — | — | 0 | |
| EXPBackbone=Nemotron-Nano-9B-v2-Base2026.05 | 5.81 | 99.8 | 42.2 | 16.2 | 37.8 | 30.6 | 45.8 | 23.6 | 0.75 | |
| KGWBackbone=Nemotron-Nano-9B-v2-Base2026.05 | 5.97 | 100 | 76.8 | 45.6 | 66.8 | 32 | 42.4 | 25.6 | 0.91 | |
| MorphMarkBackbone=Mistral-Small-3.1-24B-Base-25032026.05 | 6.08 | 99.4 | 56 | 34.2 | 45.8 | 47.6 | 51.4 | 1 | 0.32 | |
| UPVBackbone=Mistral-Small-3.1-24B-Base-25032026.05 | 6.12 | 99 | 61.4 | 23.6 | 38 | 47.2 | 51.4 | 1.4 | 0.35 | |
| PMarkBackbone=Nemotron-Nano-9B-v2-Base2026.05 | 6.22 | 96.6 | 24.2 | 28.3 | 46.3 | 26 | 49.8 | 24.2 | 1.16 | |
| KGWBackbone=Mistral-Small-3.1-24B-Base-25032026.05 | 6.65 | 100 | 64 | 34.4 | 54.4 | 38 | 50.2 | 11.8 | 0.89 | |
| SAMARKBackbone=Mistral-Small-3.1-24B-Base-2503, Optimization Status=Without Quality Optimization2026.05 | — | 98.7 | 88.7 | 89 | 90.3 | 33 | 59 | 8 | — | |
| SAMARKBackbone=Nemotron-Nano-9B-v2-Base, Optimization Status=Without Quality Optimization2026.05 | — | 99.4 | 86.6 | 84.8 | 91.2 | 20.4 | 56.4 | 23.2 | — |