# Video Question Answering on Video-Holmes Source: [Video Question Answering on Video-Holmes]() Collection index: [llms.txt]() Data updated: 2026-03-17T10:03:08.489Z Evaluation of model performance on the Video-Holmes dataset for Video Question Answering tasks, reporting the Average score. A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. | Scope | Recorded value | | --- | --- | | Task | Video Question Answering | | Dataset (including any stated split) | Video-Holmes | | Metric | Average Score | | Unit | PERCENT | | Direction | Higher is better | | Results loaded | 12 of 12 | | Coverage | All results currently returned for this benchmark | Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol. ## Metrics - [Average Score](): Higher is better; unit: PERCENT. ## Recorded results | Method | Average Score | Result date | Report paper | | --- | --- | --- | --- | | Qwen3-VL8B + OutRo | 46.76 | Not recorded | 69b8e0140636e5bf4afbea8e | | Qwen3-VL8B | 46.65 | Not recorded | 69b8e0140636e5bf4afbea8e | | Video-o3 (SFT+RL) | 46.5 | Not recorded | 69802fdda8b01e76cc8e185e | | Video-o3 (RL) | 46.1 | Not recorded | 69802fdda8b01e76cc8e185e | | Conan | 44.6 | Not recorded | 69802fdda8b01e76cc8e185e | | Rewatch-R1 | 44.3 | Not recorded | 69802fdda8b01e76cc8e185e | | Video-Thinker | 43.2 | Not recorded | 69802fdda8b01e76cc8e185e | | VideoLLaMA37B + OutRo | 42.62 | Not recorded | 69b8e0140636e5bf4afbea8e | | Qwen2.5-VL3B + OutRo | 42.41 | Not recorded | 69b8e0140636e5bf4afbea8e | | Qwen2.5-VL3B | 41.26 | Not recorded | 69b8e0140636e5bf4afbea8e | | VideoLLaMA37B | 41.15 | Not recorded | 69b8e0140636e5bf4afbea8e | | Qwen2.5-VL | 34.7 | Not recorded | 69802fdda8b01e76cc8e185e | ### Qwen3-VL8B + OutRo Average Score: 46.76. Result ID: 69b926cc47f34ce924a25670. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"Qwen3-VL8B","Decoding":"OutRo"}. This record does not assert independent reproduction. ### Qwen3-VL8B Average Score: 46.65. Result ID: 69b926cc47f34ce924a2566f. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"Qwen3-VL8B"}. This record does not assert independent reproduction. ### Video-o3 (SFT+RL) Average Score: 46.5. Result ID: 69976e975698665ec4c26dc6. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Size":"7B"}. This record does not assert independent reproduction. ### Video-o3 (RL) Average Score: 46.1. Result ID: 69976e975698665ec4c26dc5. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Size":"7B"}. This record does not assert independent reproduction. ### Conan Average Score: 44.6. Result ID: 69976e975698665ec4c26dc4. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Size":"7B"}. This record does not assert independent reproduction. ### Rewatch-R1 Average Score: 44.3. Result ID: 69976e975698665ec4c26dc2. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Size":"7B"}. This record does not assert independent reproduction. ### Video-Thinker Average Score: 43.2. Result ID: 69976e975698665ec4c26dc3. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Size":"7B"}. This record does not assert independent reproduction. ### VideoLLaMA37B + OutRo Average Score: 42.62. Result ID: 69b926cc47f34ce924a25672. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"VideoLLaMA37B","Decoding":"OutRo"}. This record does not assert independent reproduction. ### Qwen2.5-VL3B + OutRo Average Score: 42.41. Result ID: 69b926cc47f34ce924a2566e. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"Qwen2.5-VL3B","Decoding":"OutRo"}. This record does not assert independent reproduction. ### Qwen2.5-VL3B Average Score: 41.26. Result ID: 69b926cc47f34ce924a2566d. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"Qwen2.5-VL3B"}. This record does not assert independent reproduction. ### VideoLLaMA37B Average Score: 41.15. Result ID: 69b926cc47f34ce924a25671. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"VideoLLaMA37B"}. This record does not assert independent reproduction. ### Qwen2.5-VL Average Score: 34.7. Result ID: 69976e975698665ec4c26dc1. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Size":"7B"}. This record does not assert independent reproduction. Page 1 of 1. This is an index page, not the full collection.