# REKEY: Metadata-Grounded Visual-Key Regeneration for Contamination-Resilient VQA Evaluation

Source: [REKEY: Metadata-Grounded Visual-Key Regeneration for Contamination-Resilient VQA Evaluation](<https://www.sota2.com/research/paper/6a3a130a206542251d7c6f3d>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2026-06-24T05:00:04.550Z

Authors: Tengjie Lin, Yutao Sun, Jingwei Ni, Shuhan Ge, Hao-Xuan Ma, Yanting Miao, Wangyue Lu, Mingshuai Chen, Tiancheng Zhao, Jianwei Yin

Published: 2026-06-17T15:35:36.000Z

## Abstract

Static visual question answering (VQA) benchmarks age quickly: Once the items leak into training corpora, scores can reflect memorization rather than genuine visual ability, thus obscuring real progress. Rebuilding high-quality benchmarks such as V\*Bench requires substantial human annotation, yet each static release can quickly become another leaked artifact. We propose ReKey, a live benchmark protocol that randomly regenerates the answer-bearing local detail, or visual key, in real images at evaluation time. Using human-validated edit slots, ReKey samples fresh instances with new answers, construction-grounded labels, and controlled visual-search difficulty. On V\*Bench, the ReKey regenerated benchmark reveals a sharp score jump across eight frontier vision-language models (VLMs): The original items score 9.5--18.8 percentage points higher than the regenerated variants. By making the visual key renewable, ReKey keeps evaluation fresh as models and training data evolve.

## Recorded benchmark results

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

- [Visual Question Answering on V\*Bench Original](<https://www.sota2.com/research/sota/visual-question-answering-on-v-bench-original.md>): Visual Question Answering; V\*Bench Original; Accuracy: not recorded.

- [Visual Question Answering on V\*Bench Relabel](<https://www.sota2.com/research/sota/visual-question-answering-on-v-bench-relabel.md>): Visual Question Answering; V\*Bench Relabel; Accuracy: not recorded.

- [Visual Question Answering on V\*Bench (REKEY renewal)](<https://www.sota2.com/research/sota/visual-question-answering-on-v-bench-rekey-renewal.md>): Visual Question Answering; V\*Bench (REKEY renewal); Accuracy: not recorded.

## Original sources and implementation

[Original paper](<https://arxiv.org/abs/2606.20736>)

[PDF](<https://arxiv.org/pdf/2606.20736.pdf>)