# Auditing Generalization in AI-Generated Video Detection: A Six-Control Protocol and the VidAudit Toolkit Source: [Auditing Generalization in AI-Generated Video Detection: A Six-Control Protocol and the VidAudit Toolkit]() Collection index: [llms.txt]() Data updated: 2026-07-02T05:00:05.660Z Authors: Mert Onur Cakiroglu, Zhihe Lu, Mehmet Dalkilic, Hasan Kurban Published: 2026-06-30T00:29:53.000Z ## Abstract AI-generated video detection benchmarks such as GenVidBench and AIGVDBench are the de facto leaderboards, yet most evaluation protocols leave uncontrolled confounds that can inflate reported generalization. As an existence proof, a three-feature clip-length classifier reaches a leave-one-generator-out (LOGO) AUC of 0.998 on GenVidBench under unaudited evaluation, while measuring nothing about motion. A 20-paper survey finds none applying all six standard controls that would catch this, so we combine them into an audited protocol and apply it to six representative feature sources (three published detectors and three repurposed signal sources), re-running it cross-dataset on AIGVDBench. The audit both debunks and certifies: the trivial classifier collapses to near chance (0.529), a CLIP baseline is caught carrying dataset identity, and the 2025 forensic detector WaveRep clears the floor at out-of-distribution LOGO AUC 0.996 with chance-level real-vs-real coherence. At a deployable FPR of 0.1%, multiple high-AUC methods fall to single-digit recall and the leaderboard order changes, so we recommend an audited tuple (AUC, above-floor margin, operating-point recall, and calibration) over a single number. As a white-box positive control, we add TemporalSpec (codec motion vectors); via cross-substrate feature fusion (XSFF), a second substrate adds genuine complementarity that survives the audit. We release VidAudit, to our knowledge the largest unified and audited detector collection for this task, providing 14 detectors behind one plugin API, a leaderboard, and Croissant metadata, available at https://github.com/KurbanIntelligenceLab/vidaudit. Together, the protocol and toolkit move evaluation from leaderboard rank toward whether a result measures what it claims. ## Recorded benchmark results A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. - [AI-generated video detection on AIGVDBench LOGO OOD 2025-era](): AI-generated Video Detection; AIGVDBench LOGO OOD 2025-era; AUC: 68.7. - [AI-generated video detection on GenVidBench matched 27k](): AI-generated Video Detection; GenVidBench matched 27k; Audited LOGO OOD (AUC): 94.6. - [AI-generated video detection on AIGVDBench (ID)](): AI-generated Video Detection; AIGVDBench (ID); AUC: not recorded. - [AI-generated video detection on AIGVDBench LOGO ID](): AI-generated Video Detection; AIGVDBench LOGO ID; AUC: not recorded. - [AI-generated video detection on AIGVDBench LOGO OOD](): AI-generated Video Detection; AIGVDBench LOGO OOD; AUC: not recorded. ## Original sources and implementation [Original paper]() [PDF]()