# VLM-as-Judge Evaluation on ArcBench VLM-as-Judge Open Source: [VLM-as-Judge Evaluation on ArcBench VLM-as-Judge Open]() Collection index: [llms.txt]() Data updated: 2026-04-15T07:42:16.438Z Evaluation of Vision-Language Model (VLM) performance when acting as a judge using the ArcBench VLM-as-Judge Open dataset, reporting metrics such as TQ, NF, VL, and VT. A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. | Scope | Recorded value | | --- | --- | | Task | VLM-as-Judge Evaluation | | Dataset (including any stated split) | ArcBench VLM-as-Judge Open | | Metric | TQ | | Unit | PERCENT | | Direction | Higher is better | | Results loaded | 15 of 15 | | Coverage | All results currently returned for this benchmark | Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol. ## Metrics - [TQ](): Higher is better; unit: PERCENT. - [NF](): Higher is better; unit: PERCENT. - [VL](): Higher is better; unit: PERCENT. - [VT](): Higher is better; unit: PERCENT. ## Recorded results | Method | TQ | Result date | Report paper | | --- | --- | --- | --- | | HTML | 86.11 | Not recorded | 69df1b987d0172140ec7b92e | | ArcDeck | 80.06 | Not recorded | 69df1b987d0172140ec7b92e | | ArcDeck | 78.38 | Not recorded | 69df1b987d0172140ec7b92e | | SlideGen | 69.18 | Not recorded | 69df1b987d0172140ec7b92e | | SlideGen | 69.02 | Not recorded | 69df1b987d0172140ec7b92e | | HTML | 67.55 | Not recorded | 69df1b987d0172140ec7b92e | | P2Poster | 65.96 | Not recorded | 69df1b987d0172140ec7b92e | | P2Poster | 62.2 | Not recorded | 69df1b987d0172140ec7b92e | | ArcDeck | 51.68 | Not recorded | 69df1b987d0172140ec7b92e | | PPTAgent | 41.55 | Not recorded | 69df1b987d0172140ec7b92e | | PPTAgent | 35.95 | Not recorded | 69df1b987d0172140ec7b92e | | P2Poster | 32.32 | Not recorded | 69df1b987d0172140ec7b92e | | SlideGen | 19.28 | Not recorded | 69df1b987d0172140ec7b92e | | HTML | 16.72 | Not recorded | 69df1b987d0172140ec7b92e | | PPTAgent | 16.43 | Not recorded | 69df1b987d0172140ec7b92e | ### HTML TQ: 86.11. Result ID: 69df404547f34ce924c7b7ba. [Reporting paper]() Recorded evaluation configuration: {"Generation Model":"GPT-5"}. This record does not assert independent reproduction. ### ArcDeck TQ: 80.06. Result ID: 69df404547f34ce924c7b7c8. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Generation Model":"Qwen-3-VL-32B-Instruct"}. This record does not assert independent reproduction. ### ArcDeck TQ: 78.38. Result ID: 69df404547f34ce924c7b7be. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Generation Model":"GPT-5"}. This record does not assert independent reproduction. ### SlideGen TQ: 69.18. Result ID: 69df404547f34ce924c7b7c7. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Generation Model":"Qwen-3-VL-32B-Instruct"}. This record does not assert independent reproduction. ### SlideGen TQ: 69.02. Result ID: 69df404547f34ce924c7b7bd. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Generation Model":"GPT-5"}. This record does not assert independent reproduction. ### HTML TQ: 67.55. Result ID: 69df404547f34ce924c7b7c4. [Reporting paper]() Recorded evaluation configuration: {"Generation Model":"Qwen-3-VL-32B-Instruct"}. This record does not assert independent reproduction. ### P2Poster TQ: 65.96. Result ID: 69df404547f34ce924c7b7c5. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Generation Model":"Qwen-3-VL-32B-Instruct"}. This record does not assert independent reproduction. ### P2Poster TQ: 62.2. Result ID: 69df404547f34ce924c7b7bb. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Generation Model":"GPT-5"}. This record does not assert independent reproduction. ### ArcDeck TQ: 51.68. Result ID: 69df404547f34ce924c7b7c3. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Generation Model":"GPT-4o"}. This record does not assert independent reproduction. ### PPTAgent TQ: 41.55. Result ID: 69df404547f34ce924c7b7c6. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Generation Model":"Qwen-3-VL-32B-Instruct"}. This record does not assert independent reproduction. ### PPTAgent TQ: 35.95. Result ID: 69df404547f34ce924c7b7bc. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Generation Model":"GPT-5"}. This record does not assert independent reproduction. ### P2Poster TQ: 32.32. Result ID: 69df404547f34ce924c7b7c0. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Generation Model":"GPT-4o"}. This record does not assert independent reproduction. ### SlideGen TQ: 19.28. Result ID: 69df404547f34ce924c7b7c2. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Generation Model":"GPT-4o"}. This record does not assert independent reproduction. ### HTML TQ: 16.72. Result ID: 69df404547f34ce924c7b7bf. [Reporting paper]() Recorded evaluation configuration: {"Generation Model":"GPT-4o"}. This record does not assert independent reproduction. ### PPTAgent TQ: 16.43. Result ID: 69df404547f34ce924c7b7c1. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Generation Model":"GPT-4o"}. This record does not assert independent reproduction. Page 1 of 1. This is an index page, not the full collection.