# VLM-based Q/A Quiz on ArcBench VLM-based Q/A Quiz Open Source: [VLM-based Q/A Quiz on ArcBench VLM-based Q/A Quiz Open]() Collection index: [llms.txt]() Data updated: 2026-04-15T07:42:16.385Z Evaluation of model performance on the ArcBench VLM-based Q/A Quiz Open dataset for VLM-based Question Answering, reporting metrics across Story, Visuals, Hard, and Depth categories. A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. | Scope | Recorded value | | --- | --- | | Task | VLM-based Q/A Quiz | | Dataset (including any stated split) | ArcBench VLM-based Q/A Quiz Open | | Metric | Accuracy (Story) | | Unit | PERCENT | | Direction | Higher is better | | Results loaded | 15 of 15 | | Coverage | All results currently returned for this benchmark | Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol. ## Metrics - [Accuracy (Story)](): Higher is better; unit: PERCENT. - [Accuracy (Visuals)](): Higher is better; unit: PERCENT. - [Accuracy (Hard)](): Higher is better; unit: PERCENT. - [Accuracy (Depth)](): Higher is better; unit: PERCENT. ## Recorded results | Method | Accuracy (Story) | Result date | Report paper | | --- | --- | --- | --- | | ArcDeck | 95.69 | Not recorded | 69df1b987d0172140ec7b92e | | ArcDeck | 95.46 | Not recorded | 69df1b987d0172140ec7b92e | | ArcDeck | 95.41 | Not recorded | 69df1b987d0172140ec7b92e | | SlideGen | 95.26 | Not recorded | 69df1b987d0172140ec7b92e | | P2Poster | 95.22 | Not recorded | 69df1b987d0172140ec7b92e | | HTML | 95.16 | Not recorded | 69df1b987d0172140ec7b92e | | SlideGen | 94.81 | Not recorded | 69df1b987d0172140ec7b92e | | P2Poster | 94.28 | Not recorded | 69df1b987d0172140ec7b92e | | P2Poster | 94.12 | Not recorded | 69df1b987d0172140ec7b92e | | HTML | 94.07 | Not recorded | 69df1b987d0172140ec7b92e | | SlideGen | 93.75 | Not recorded | 69df1b987d0172140ec7b92e | | PPTAgent | 93.35 | Not recorded | 69df1b987d0172140ec7b92e | | HTML | 93.34 | Not recorded | 69df1b987d0172140ec7b92e | | PPTAgent | 92.96 | Not recorded | 69df1b987d0172140ec7b92e | | PPTAgent | 92.66 | Not recorded | 69df1b987d0172140ec7b92e | ### ArcDeck Accuracy (Story): 95.69. Result ID: 69df404547f34ce924c7b7aa. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Generation Model":"Qwen-3-VL-32B-Instruct"}. This record does not assert independent reproduction. ### ArcDeck Accuracy (Story): 95.46. Result ID: 69df404547f34ce924c7b7a5. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Generation Model":"GPT-4o"}. This record does not assert independent reproduction. ### ArcDeck Accuracy (Story): 95.41. Result ID: 69df404547f34ce924c7b7a0. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Generation Model":"GPT-5"}. This record does not assert independent reproduction. ### SlideGen Accuracy (Story): 95.26. Result ID: 69df404547f34ce924c7b79f. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Generation Model":"GPT-5"}. This record does not assert independent reproduction. ### P2Poster Accuracy (Story): 95.22. Result ID: 69df404547f34ce924c7b79d. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Generation Model":"GPT-5"}. This record does not assert independent reproduction. ### HTML Accuracy (Story): 95.16. Result ID: 69df404547f34ce924c7b79c. [Reporting paper]() Recorded evaluation configuration: {"Generation Model":"GPT-5"}. This record does not assert independent reproduction. ### SlideGen Accuracy (Story): 94.81. Result ID: 69df404547f34ce924c7b7a9. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Generation Model":"Qwen-3-VL-32B-Instruct"}. This record does not assert independent reproduction. ### P2Poster Accuracy (Story): 94.28. Result ID: 69df404547f34ce924c7b7a7. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Generation Model":"Qwen-3-VL-32B-Instruct"}. This record does not assert independent reproduction. ### P2Poster Accuracy (Story): 94.12. Result ID: 69df404547f34ce924c7b7a2. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Generation Model":"GPT-4o"}. This record does not assert independent reproduction. ### HTML Accuracy (Story): 94.07. Result ID: 69df404547f34ce924c7b7a6. [Reporting paper]() Recorded evaluation configuration: {"Generation Model":"Qwen-3-VL-32B-Instruct"}. This record does not assert independent reproduction. ### SlideGen Accuracy (Story): 93.75. Result ID: 69df404547f34ce924c7b7a4. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Generation Model":"GPT-4o"}. This record does not assert independent reproduction. ### PPTAgent Accuracy (Story): 93.35. Result ID: 69df404547f34ce924c7b7a3. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Generation Model":"GPT-4o"}. This record does not assert independent reproduction. ### HTML Accuracy (Story): 93.34. Result ID: 69df404547f34ce924c7b7a1. [Reporting paper]() Recorded evaluation configuration: {"Generation Model":"GPT-4o"}. This record does not assert independent reproduction. ### PPTAgent Accuracy (Story): 92.96. Result ID: 69df404547f34ce924c7b79e. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Generation Model":"GPT-5"}. This record does not assert independent reproduction. ### PPTAgent Accuracy (Story): 92.66. Result ID: 69df404547f34ce924c7b7a8. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Generation Model":"Qwen-3-VL-32B-Instruct"}. This record does not assert independent reproduction. Page 1 of 1. This is an index page, not the full collection.