Watermark Detection and Quality Evaluation on Aggregate (MMLU, HellaSwag, ARC-C, GPQA, MBPP, GSM8K)
0.99TPR@1DLM watermark
Evaluation Results
| Method | Links | ||||
|---|---|---|---|---|---|
| DLM watermarkModel=LLADA-8B, Context C={-1}, Strength δ=4, Generation Temperature=0.5, Benchmark Temperature=0.12025.09 | 0.99 | 1.9 | 8.43 | 56 | |
| DLM watermarkModel=LLADA-8B, Context C={-1, 1}, Strength δ=5, Generation Temperature=0.5, Benchmark Temperature=0.12025.09 | 0.99 | 1.8 | 8.6 | 53.7 | |
| DLM watermarkModel=LLADA-8B, Context C={-2, -1}, Strength δ=5, Generation Temperature=0.5, Benchmark Temperature=0.12025.09 | 0.99 | 1.8 | 8.59 | 54.5 | |
| DLM watermarkModel=DREAM-7B, Context C={-1}, Strength δ=4, Generation Temperature=0.5, Benchmark Temperature=0.12025.09 | 0.99 | 2.32 | 7.76 | 50.1 | |
| DLM watermarkModel=DREAM-7B, Context C={-1, 1}, Strength δ=5, Generation Temperature=0.5, Benchmark Temperature=0.12025.09 | 0.99 | 2.18 | 7.85 | 47 | |
| DLM watermarkModel=DREAM-7B, Context C={-2, -1}, Strength δ=5, Generation Temperature=0.5, Benchmark Temperature=0.12025.09 | 0.99 | 2.15 | 7.9 | 47.6 | |
| BaselineModel=LLADA-8B, Context C={-2, -1}, Strength δ=5, Generation Temperature=0.5, Benchmark Temperature=0.12025.09 | 0.83 | 1.94 | 8.37 | 58.5 | |
| BaselineModel=DREAM-7B, Context C={-1, 1}, Strength δ=5, Generation Temperature=0.5, Benchmark Temperature=0.12025.09 | 0.74 | 2.18 | 7.94 | 35.5 | |
| BaselineModel=DREAM-7B, Context C={-2, -1}, Strength δ=5, Generation Temperature=0.5, Benchmark Temperature=0.12025.09 | 0.7 | 2.23 | 8.2 | 32.9 | |
| BaselineModel=LLADA-8B, Context C={-1, 1}, Strength δ=5, Generation Temperature=0.5, Benchmark Temperature=0.12025.09 | 0.69 | 1.86 | 8.51 | 55.7 | |
| BaselineModel=LLADA-8B, Context C={-1}, Strength δ=4, Generation Temperature=0.5, Benchmark Temperature=0.12025.09 | 0.63 | 1.93 | 8.48 | 55.5 | |
| BaselineModel=DREAM-7B, Context C={-1}, Strength δ=4, Generation Temperature=0.5, Benchmark Temperature=0.12025.09 | 0.49 | 2.27 | 7.95 | 35.8 | |
| UnwatermarkedModel=LLADA-8B, Context C={-1}, Strength δ=4, Generation Temperature=0.5, Benchmark Temperature=0.12025.09 | 0 | 1.56 | 8.95 | 59.4 | |
| UnwatermarkedModel=LLADA-8B, Context C={-1, 1}, Strength δ=5, Generation Temperature=0.5, Benchmark Temperature=0.12025.09 | 0 | 1.56 | 8.95 | 59.4 | |
| UnwatermarkedModel=LLADA-8B, Context C={-2, -1}, Strength δ=5, Generation Temperature=0.5, Benchmark Temperature=0.12025.09 | 0 | 1.56 | 8.95 | 59.4 | |
| UnwatermarkedModel=DREAM-7B, Context C={-1}, Strength δ=4, Generation Temperature=0.5, Benchmark Temperature=0.12025.09 | 0 | 1.94 | 8.45 | 50.4 | |
| UnwatermarkedModel=DREAM-7B, Context C={-1, 1}, Strength δ=5, Generation Temperature=0.5, Benchmark Temperature=0.12025.09 | 0 | 1.94 | 8.45 | 50.4 | |
| UnwatermarkedModel=DREAM-7B, Context C={-2, -1}, Strength δ=5, Generation Temperature=0.5, Benchmark Temperature=0.12025.09 | 0 | 1.94 | 8.45 | 50.4 |