# Open-ended Video Question Answering on VideoChatGPT Bench (test)

Source: [Open-ended Video Question Answering on VideoChatGPT Bench (test)](<https://www.sota2.com/research/sota/open-ended-video-question-answering-on-videochatgpt-bench-test>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2026-02-26T05:39:54.642Z

Evaluation of model performance on the test split of the VideoChatGPT Bench for Open-ended Video Question Answering, reporting scores for Correctness of Information (CI), Detail Orientation (DO), Contextual Understanding (CU), Temporal Understanding (TU), Consistency (CO), and Average Score (Avg).

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

| Scope | Recorded value |
| --- | --- |
| Task | Open-ended Video Question Answering |
| Dataset (including any stated split) | VideoChatGPT Bench (test) |
| Metric | Correctness of Information (CI) |
| Unit | SCALAR |
| Direction | Higher is better |
| Results loaded | 15 of 15 |
| Coverage | All results currently returned for this benchmark |

Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol.

## Metrics

- [Correctness of Information (CI)](<https://www.sota2.com/research/sota/open-ended-video-question-answering-on-videochatgpt-bench-test.md?metric=Correctness%20of%20Information%20(CI)>): Higher is better; unit: SCALAR.

- [Detail Orientation (DO)](<https://www.sota2.com/research/sota/open-ended-video-question-answering-on-videochatgpt-bench-test.md?metric=Detail%20Orientation%20(DO)>): Higher is better; unit: SCALAR.

- [Contextual Understanding (CU)](<https://www.sota2.com/research/sota/open-ended-video-question-answering-on-videochatgpt-bench-test.md?metric=Contextual%20Understanding%20(CU)>): Higher is better; unit: SCALAR.

- [Temporal Understanding (TU)](<https://www.sota2.com/research/sota/open-ended-video-question-answering-on-videochatgpt-bench-test.md?metric=Temporal%20Understanding%20(TU)>): Higher is better; unit: SCALAR.

- [Consistency (CO)](<https://www.sota2.com/research/sota/open-ended-video-question-answering-on-videochatgpt-bench-test.md?metric=Consistency%20(CO)>): Higher is better; unit: SCALAR.

- [Average Score (Avg)](<https://www.sota2.com/research/sota/open-ended-video-question-answering-on-videochatgpt-bench-test.md?metric=Average%20Score%20(Avg)>): Higher is better; unit: SCALAR.

## Recorded results

| Method | Correctness of Information (CI) | Result date | Report paper |
| --- | --- | --- | --- |
| LLaVA-OneVision | 3.45 | Not recorded | 68fa77801e209131cea739ee |
| LLaVA-Next-Video | 3.39 | Not recorded | 68fa77801e209131cea739ee |
| HICom | 3.33 | Not recorded | 68fa77801e209131cea739ee |
| HICom | 3.32 | Not recorded | 68fa77801e209131cea739ee |
| HICom | 3.29 | Not recorded | 68fa77801e209131cea739ee |
| ST-LLM | 3.23 | Not recorded | 68fa77801e209131cea739ee |
| PLLAVA | 3.21 | Not recorded | 68fa77801e209131cea739ee |
| VideoLLaMA2 | 3.16 | Not recorded | 68fa77801e209131cea739ee |
| SF-LLAVA | 3.09 | Not recorded | 68fa77801e209131cea739ee |
| HICom | 3.09 | Not recorded | 68fa77801e209131cea739ee |
| LongVA | 3.05 | Not recorded | 68fa77801e209131cea739ee |
| VideoChat2 | 3.02 | Not recorded | 68fa77801e209131cea739ee |
| LLaMA-VID | 2.96 | Not recorded | 68fa77801e209131cea739ee |
| Chat-Univi | 2.89 | Not recorded | 68fa77801e209131cea739ee |
| LongVLM | 2.76 | Not recorded | 68fa77801e209131cea739ee |

### LLaVA-OneVision

Correctness of Information (CI): 3.45. Result ID: 699498333a6c3148491e42cb.

[Reporting paper](<https://www.sota2.com/research/paper/68fa77801e209131cea739ee.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa756f1e209131cea67e58.md>)

[Implementation](<https://llava-vl.github.io/blog/llava-onevision>)

Recorded evaluation configuration: {"# Frames":"32","# Tokens":"6272"}. This record does not assert independent reproduction.

### LLaVA-Next-Video

Correctness of Information (CI): 3.39. Result ID: 699498333a6c3148491e42c7.

[Reporting paper](<https://www.sota2.com/research/paper/68fa77801e209131cea739ee.md>)

Recorded evaluation configuration: {"# Frames":"32","# Tokens":"4608"}. This record does not assert independent reproduction.

### HICom

Correctness of Information (CI): 3.33. Result ID: 699498333a6c3148491e42cf.

[Reporting paper](<https://www.sota2.com/research/paper/68fa77801e209131cea739ee.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa77801e209131cea739ee.md>)

[Implementation](<https://github.com/lntzm/HICom>)

Recorded evaluation configuration: {"Model Scale":"7B","# Frames":"128","# Tokens":"2624"}. This record does not assert independent reproduction.

### HICom

Correctness of Information (CI): 3.32. Result ID: 699498333a6c3148491e42ce.

[Reporting paper](<https://www.sota2.com/research/paper/68fa77801e209131cea739ee.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa77801e209131cea739ee.md>)

[Implementation](<https://github.com/lntzm/HICom>)

Recorded evaluation configuration: {"Model Scale":"7B","# Frames":"64","# Tokens":"1328"}. This record does not assert independent reproduction.

### HICom

Correctness of Information (CI): 3.29. Result ID: 699498333a6c3148491e42cd.

[Reporting paper](<https://www.sota2.com/research/paper/68fa77801e209131cea739ee.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa77801e209131cea739ee.md>)

[Implementation](<https://github.com/lntzm/HICom>)

Recorded evaluation configuration: {"Model Scale":"7B","# Frames":"32","# Tokens":"680"}. This record does not assert independent reproduction.

### ST-LLM

Correctness of Information (CI): 3.23. Result ID: 699498333a6c3148491e42c5.

[Reporting paper](<https://www.sota2.com/research/paper/68fa77801e209131cea739ee.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa72bd1e209131cea584a2.md>)

[Implementation](<https://github.com/TencentARC/ST-LLM>)

Recorded evaluation configuration: {"# Frames":"16","# Tokens":"512"}. This record does not assert independent reproduction.

### PLLAVA

Correctness of Information (CI): 3.21. Result ID: 699498333a6c3148491e42c6.

[Reporting paper](<https://www.sota2.com/research/paper/68fa77801e209131cea739ee.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa73101e209131cea5a3ea.md>)

[Implementation](<https://pllava.github.io/>)

Recorded evaluation configuration: {"# Frames":"16","# Tokens":"2304"}. This record does not assert independent reproduction.

### VideoLLaMA2

Correctness of Information (CI): 3.16. Result ID: 699498333a6c3148491e42c9.

[Reporting paper](<https://www.sota2.com/research/paper/68fa77801e209131cea739ee.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa75761e209131cea681dd.md>)

[Implementation](<https://github.com/DAMO-NLP-SG/VideoLLaMA2>)

Recorded evaluation configuration: {"# Frames":"16","# Tokens":"1152"}. This record does not assert independent reproduction.

### SF-LLAVA

Correctness of Information (CI): 3.09. Result ID: 699498333a6c3148491e42ca.

[Reporting paper](<https://www.sota2.com/research/paper/68fa77801e209131cea739ee.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa74c91e209131cea63cb0.md>)

[Implementation](<https://huggingface.co/collections/liuhaotian/llava-next-65b9e40155f60fd046a5ccf2>)

Recorded evaluation configuration: {"# Frames":"50","# Tokens":"3680"}. This record does not assert independent reproduction.

### HICom

Correctness of Information (CI): 3.09. Result ID: 699498333a6c3148491e42cc.

[Reporting paper](<https://www.sota2.com/research/paper/68fa77801e209131cea739ee.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa77801e209131cea739ee.md>)

[Implementation](<https://github.com/lntzm/HICom>)

Recorded evaluation configuration: {"Model Scale":"1.5B","# Frames":"32","# Tokens":"680"}. This record does not assert independent reproduction.

### LongVA

Correctness of Information (CI): 3.05. Result ID: 699498333a6c3148491e42c8.

[Reporting paper](<https://www.sota2.com/research/paper/68fa77801e209131cea739ee.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa73ea1e209131cea5edfe.md>)

[Implementation](<https://github.com/EvolvingLMMs-Lab/LongVA>)

Recorded evaluation configuration: {"# Frames":"128","# Tokens":"18432"}. This record does not assert independent reproduction.

### VideoChat2

Correctness of Information (CI): 3.02. Result ID: 699498333a6c3148491e42c1.

[Reporting paper](<https://www.sota2.com/research/paper/68fa77801e209131cea739ee.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa73411e209131cea5b4b7.md>)

Recorded evaluation configuration: {"# Frames":"16","# Tokens":"1536"}. This record does not assert independent reproduction.

### LLaMA-VID

Correctness of Information (CI): 2.96. Result ID: 699498333a6c3148491e42c3.

[Reporting paper](<https://www.sota2.com/research/paper/68fa77801e209131cea739ee.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa71461e209131cea50559.md>)

[Implementation](<https://github.com/dvlab-research/LLaMA-VID>)

Recorded evaluation configuration: {"# Frames":"1fps","# Tokens":"2tps"}. This record does not assert independent reproduction.

### Chat-Univi

Correctness of Information (CI): 2.89. Result ID: 699498333a6c3148491e42c2.

[Reporting paper](<https://www.sota2.com/research/paper/68fa77801e209131cea739ee.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa72d31e209131cea58bff.md>)

[Implementation](<https://github.com/PKU-YuanGroup/Chat-UniVi>)

Recorded evaluation configuration: {"# Frames":"64","# Tokens":"448"}. This record does not assert independent reproduction.

### LongVLM

Correctness of Information (CI): 2.76. Result ID: 699498333a6c3148491e42c4.

[Reporting paper](<https://www.sota2.com/research/paper/68fa77801e209131cea739ee.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa74361e209131cea6077a.md>)

Recorded evaluation configuration: {"# Frames":"100","# Tokens":"305"}. This record does not assert independent reproduction.

Page 1 of 1. This is an index page, not the full collection.