# Finding needles in a haystack: A Black-Box Approach to Invisible Watermark Detection Source: [Finding needles in a haystack: A Black-Box Approach to Invisible Watermark Detection]() Collection index: [llms.txt]() Data updated: 2025-10-26T02:33:49.815Z Authors: Minzhou Pan, Zhenting Wang, Xin Dong, Vikash Sehwag, Lingjuan Lyu, Xue Lin Published: 2024-03-23T23:22:54.000Z ## Abstract In this paper, we propose WaterMark Detection (WMD), the first invisible watermark detection method under a black-box and annotation-free setting. WMD is capable of detecting arbitrary watermarks within a given reference dataset using a clean non-watermarked dataset as a reference, without relying on specific decoding methods or prior knowledge of the watermarking techniques. We develop WMD using foundations of offset learning, where a clean non-watermarked dataset enables us to isolate the influence of only watermarked samples in the reference dataset. Our comprehensive evaluations demonstrate the effectiveness of WMD, significantly outperforming naive detection methods, which only yield AUC scores around 0.5. In contrast, WMD consistently achieves impressive detection AUC scores, surpassing 0.9 in most single-watermark datasets and exceeding 0.7 in more challenging multi-watermark scenarios across diverse datasets and watermarking methods. As invisible watermarks become increasingly prevalent, while specific decoding techniques remain undisclosed, our approach provides a versatile solution and establishes a path toward increasing accountability, transparency, and trust in our digital visual content. ## Recorded benchmark results A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. - [Audio Watermark Detection on VoxWatermark 2 (test)](): Audio Watermark Detection; VoxWatermark 2 (test); TPR: 100. - [Audio Watermark Detection on VoxWatermark Set 1 (T1) (test)](): Audio Watermark Detection; VoxWatermark Set 1 (T1) (test); TPR: 96.09. - [Audio Watermark Detection on VoxWatermark (val)](): Audio Watermark Detection; VoxWatermark (val); Overall AUROC: 72. - [Audio Watermark Detection on VoxWatermark (test 2)](): Audio Watermark Detection; VoxWatermark (test 2); Overall AUROC: 57.9. - [Audio Watermark Detection on VoxWatermark T2 (test)](): Audio Watermark Detection; VoxWatermark T2 (test); AUROC: 57.94. - [Watermark Detection on VoxPopuli-100k Scenario T1](): Watermark Detection; VoxPopuli-100k Scenario T1; AUC: 57.09. ## Original sources and implementation [Original paper]() [PDF]()