Code Watermark Detection on HumanEval+
0.869AUROCSTONE
Evaluation results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| STONEAttack=No attack, Post-hoc=No, Payload=0 bits, Training=LLM access, Generator Model=Qwen2.5-Coder-3B2026.07 | 0.869 | 42.8 | 60.4 | — | — | |
| STONEAttack=Rename 2, Post-hoc=No, Payload=0 bits, Training=LLM access, Generator Model=Qwen2.5-Coder-3B2026.07 | 0.863 | 40.6 | 56.8 | — | — | |
| STONEAttack=Rename 4, Post-hoc=No, Payload=0 bits, Training=LLM access, Generator Model=Qwen2.5-Coder-3B2026.07 | 0.856 | 39.6 | 55.7 | — | — | |
| STONEAttack=Rename 8, Post-hoc=No, Payload=0 bits, Training=LLM access, Generator Model=Qwen2.5-Coder-3B2026.07 | 0.855 | 39.5 | 55.4 | — | — | |
| STONEAttack=equality_dir, Post-hoc=No, Payload=0 bits, Training=LLM access, Generator Model=Qwen2.5-Coder-3B2026.07 | 0.851 | 39.5 | 56.2 | — | — | |
| STONEAttack=aug_assign, Post-hoc=No, Payload=0 bits, Training=LLM access, Generator Model=Qwen2.5-Coder-3B2026.07 | 0.85 | 39.6 | 56.2 | — | — | |
| STONEAttack=compare_dir, Post-hoc=No, Payload=0 bits, Training=LLM access, Generator Model=Qwen2.5-Coder-3B2026.07 | 0.85 | 39.5 | 56.2 | — | — | |
| STONEAttack=list_comp, Post-hoc=No, Payload=0 bits, Training=LLM access, Generator Model=Qwen2.5-Coder-3B2026.07 | 0.85 | 39.6 | 56.2 | — | — | |
| STONEAttack=ternary, Post-hoc=No, Payload=0 bits, Training=LLM access, Generator Model=Qwen2.5-Coder-3B2026.07 | 0.85 | 39.6 | 56.2 | — | — | |
| STONEAttack=operand_order, Post-hoc=No, Payload=0 bits, Training=LLM access, Generator Model=Qwen2.5-Coder-3B2026.07 | 0.85 | 39.6 | 56.3 | — | — | |
| STONEAttack=empty_coll, Post-hoc=No, Payload=0 bits, Training=LLM access, Generator Model=Qwen2.5-Coder-3B2026.07 | 0.85 | 39.6 | 56.2 | — | — | |
| STONEAttack=none_check, Post-hoc=No, Payload=0 bits, Training=LLM access, Generator Model=Qwen2.5-Coder-3B2026.07 | 0.85 | 39.6 | 56.2 | — | — | |
| multi-channel spread-spectrum watermarkingAttack=No attack, Post-hoc=Yes, Payload=24 bits, Training=None, Parameters=D56, n=250, t=82026.07 | — | — | — | 100 | 0 | |
| multi-channel spread-spectrum watermarkingAttack=Rename 2, Post-hoc=Yes, Payload=24 bits, Training=None, Parameters=D56, n=250, t=82026.07 | — | — | — | 98 | 0 | |
| multi-channel spread-spectrum watermarkingAttack=Rename 4, Post-hoc=Yes, Payload=24 bits, Training=None, Parameters=D56, n=250, t=82026.07 | — | — | — | 98 | 0 | |
| multi-channel spread-spectrum watermarkingAttack=Rename 8, Post-hoc=Yes, Payload=24 bits, Training=None, Parameters=D56, n=250, t=82026.07 | — | — | — | 97.6 | 0 | |
| multi-channel spread-spectrum watermarkingAttack=aug_assign, Post-hoc=Yes, Payload=24 bits, Training=None, Parameters=D56, n=250, t=82026.07 | — | — | — | 70.9 | 0 | |
| multi-channel spread-spectrum watermarkingAttack=compare_dir, Post-hoc=Yes, Payload=24 bits, Training=None, Parameters=D56, n=250, t=82026.07 | — | — | — | 59.8 | 0 | |
| multi-channel spread-spectrum watermarkingAttack=equality_dir, Post-hoc=Yes, Payload=24 bits, Training=None, Parameters=D56, n=250, t=82026.07 | — | — | — | 49.8 | 0 | |
| multi-channel spread-spectrum watermarkingAttack=list_comp, Post-hoc=Yes, Payload=24 bits, Training=None, Parameters=D56, n=250, t=82026.07 | — | — | — | 76.5 | 0 | |
| multi-channel spread-spectrum watermarkingAttack=ternary, Post-hoc=Yes, Payload=24 bits, Training=None, Parameters=D56, n=250, t=82026.07 | — | — | — | 95.6 | 0 | |
| multi-channel spread-spectrum watermarkingAttack=operand_order, Post-hoc=Yes, Payload=24 bits, Training=None, Parameters=D56, n=250, t=82026.07 | — | — | — | 29.9 | 0 | |
| multi-channel spread-spectrum watermarkingAttack=empty_coll, Post-hoc=Yes, Payload=24 bits, Training=None, Parameters=D56, n=250, t=82026.07 | — | — | — | 74.5 | 0 | |
| multi-channel spread-spectrum watermarkingAttack=none_check, Post-hoc=Yes, Payload=24 bits, Training=None, Parameters=D56, n=250, t=82026.07 | — | — | — | 97.2 | 0 |