# Video Understanding on VideoMME (Performance, Latency, and Memory)

Source: [Video Understanding on VideoMME (Performance, Latency, and Memory)](<https://www.sota2.com/research/sota/video-understanding-on-videomme-performance-latency-and-memory>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2026-05-15T08:26:52.453Z

Evaluation of video understanding capabilities on the VideoMME dataset, reporting core performance scores alongside system metrics including prefill time, sample latency, and GPU memory usage.

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

| Scope | Recorded value |
| --- | --- |
| Task | Video Understanding |
| Dataset (including any stated split) | VideoMME |
| Metric | VideoMME Performance |
| Unit | PERCENT |
| Direction | Higher is better |
| Results loaded | 14 of 14 |
| Coverage | All results currently returned for this benchmark |

Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol.

## Metrics

- [VideoMME Performance](<https://www.sota2.com/research/sota/video-understanding-on-videomme-performance-latency-and-memory.md?metric=VideoMME%20Performance>): Higher is better; unit: PERCENT.

- [Avg Delta Performance](<https://www.sota2.com/research/sota/video-understanding-on-videomme-performance-latency-and-memory.md?metric=Avg%20Delta%20Performance>): Higher is better; unit: SCALAR.

- [Prefill Time (s)](<https://www.sota2.com/research/sota/video-understanding-on-videomme-performance-latency-and-memory.md?metric=Prefill%20Time%20(s)>): Lower is better; unit: TIME.

- [Sample Latency (s)](<https://www.sota2.com/research/sota/video-understanding-on-videomme-performance-latency-and-memory.md?metric=Sample%20Latency%20(s)>): Lower is better; unit: TIME.

- [GPU Memory (GB)](<https://www.sota2.com/research/sota/video-understanding-on-videomme-performance-latency-and-memory.md?metric=GPU%20Memory%20(GB)>): Lower is better; unit: SIZE.

## Recorded results

| Method | VideoMME Performance | Result date | Report paper |
| --- | --- | --- | --- |
| OmniDrop | 66.52 | Not recorded | 6a06a876dbc836b87d829f24 |
| OmniDrop | 66.44 | Not recorded | 6a06a876dbc836b87d829f24 |
| OmniZip | 65.85 | Not recorded | 6a06a876dbc836b87d829f24 |
| DASH | 65.67 | Not recorded | 6a06a876dbc836b87d829f24 |
| DASH | 64.96 | Not recorded | 6a06a876dbc836b87d829f24 |
| OmniZip | 64.85 | Not recorded | 6a06a876dbc836b87d829f24 |
| Qwen2.5-Omni-7B | 64.67 | Not recorded | 6a06a876dbc836b87d829f24 |
| OmniDrop | 63.07 | Not recorded | 6a06a876dbc836b87d829f24 |
| OmniDrop | 63.07 | Not recorded | 6a06a876dbc836b87d829f24 |
| OmniZip | 62.52 | Not recorded | 6a06a876dbc836b87d829f24 |
| Qwen2.5-Omni-3B | 62.51 | Not recorded | 6a06a876dbc836b87d829f24 |
| DASH | 62.26 | Not recorded | 6a06a876dbc836b87d829f24 |
| OmniZip | 61.59 | Not recorded | 6a06a876dbc836b87d829f24 |
| DASH | 60.96 | Not recorded | 6a06a876dbc836b87d829f24 |

### OmniDrop

VideoMME Performance: 66.52. Result ID: 6a06d88d47f34ce924ee7223.

[Reporting paper](<https://www.sota2.com/research/paper/6a06a876dbc836b87d829f24.md>)

[Method origin paper](<https://www.sota2.com/research/paper/6a06a876dbc836b87d829f24.md>)

Recorded evaluation configuration: {"Model Backbone":"Qwen2.5-Omni-7B","Retained Ratio (%)":"30"}. This record does not assert independent reproduction.

### OmniDrop

VideoMME Performance: 66.44. Result ID: 6a06d88d47f34ce924ee7224.

[Reporting paper](<https://www.sota2.com/research/paper/6a06a876dbc836b87d829f24.md>)

[Method origin paper](<https://www.sota2.com/research/paper/6a06a876dbc836b87d829f24.md>)

Recorded evaluation configuration: {"Model Backbone":"Qwen2.5-Omni-7B","Retained Ratio (%)":"20"}. This record does not assert independent reproduction.

### OmniZip

VideoMME Performance: 65.85. Result ID: 6a06d88d47f34ce924ee721f.

[Reporting paper](<https://www.sota2.com/research/paper/6a06a876dbc836b87d829f24.md>)

[Method origin paper](<https://www.sota2.com/research/paper/69a125012d603e6de3e3c43b.md>)

Recorded evaluation configuration: {"Model Backbone":"Qwen2.5-Omni-7B","Retained Ratio (%)":"30"}. This record does not assert independent reproduction.

### DASH

VideoMME Performance: 65.67. Result ID: 6a06d88d47f34ce924ee7221.

[Reporting paper](<https://www.sota2.com/research/paper/6a06a876dbc836b87d829f24.md>)

[Method origin paper](<https://www.sota2.com/research/paper/69ba316d0636e5bf4afc0614.md>)

Recorded evaluation configuration: {"Model Backbone":"Qwen2.5-Omni-7B","Retained Ratio (%)":"30"}. This record does not assert independent reproduction.

### DASH

VideoMME Performance: 64.96. Result ID: 6a06d88d47f34ce924ee7222.

[Reporting paper](<https://www.sota2.com/research/paper/6a06a876dbc836b87d829f24.md>)

[Method origin paper](<https://www.sota2.com/research/paper/69ba316d0636e5bf4afc0614.md>)

Recorded evaluation configuration: {"Model Backbone":"Qwen2.5-Omni-7B","Retained Ratio (%)":"20"}. This record does not assert independent reproduction.

### OmniZip

VideoMME Performance: 64.85. Result ID: 6a06d88d47f34ce924ee7220.

[Reporting paper](<https://www.sota2.com/research/paper/6a06a876dbc836b87d829f24.md>)

[Method origin paper](<https://www.sota2.com/research/paper/69a125012d603e6de3e3c43b.md>)

Recorded evaluation configuration: {"Model Backbone":"Qwen2.5-Omni-7B","Retained Ratio (%)":"20"}. This record does not assert independent reproduction.

### Qwen2.5-Omni-7B

VideoMME Performance: 64.67. Result ID: 6a06d88d47f34ce924ee721e.

[Reporting paper](<https://www.sota2.com/research/paper/6a06a876dbc836b87d829f24.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa77a21e209131cea7482c.md>)

Recorded evaluation configuration: {"Model Backbone":"Qwen2.5-Omni-7B","Retained Ratio (%)":"100"}. This record does not assert independent reproduction.

### OmniDrop

VideoMME Performance: 63.07. Result ID: 6a06d88d47f34ce924ee722a.

[Reporting paper](<https://www.sota2.com/research/paper/6a06a876dbc836b87d829f24.md>)

[Method origin paper](<https://www.sota2.com/research/paper/6a06a876dbc836b87d829f24.md>)

Recorded evaluation configuration: {"Model Backbone":"Qwen2.5-Omni-3B","Retained Ratio (%)":"30"}. This record does not assert independent reproduction.

### OmniDrop

VideoMME Performance: 63.07. Result ID: 6a06d88d47f34ce924ee722b.

[Reporting paper](<https://www.sota2.com/research/paper/6a06a876dbc836b87d829f24.md>)

[Method origin paper](<https://www.sota2.com/research/paper/6a06a876dbc836b87d829f24.md>)

Recorded evaluation configuration: {"Model Backbone":"Qwen2.5-Omni-3B","Retained Ratio (%)":"20"}. This record does not assert independent reproduction.

### OmniZip

VideoMME Performance: 62.52. Result ID: 6a06d88d47f34ce924ee7226.

[Reporting paper](<https://www.sota2.com/research/paper/6a06a876dbc836b87d829f24.md>)

[Method origin paper](<https://www.sota2.com/research/paper/69a125012d603e6de3e3c43b.md>)

Recorded evaluation configuration: {"Model Backbone":"Qwen2.5-Omni-3B","Retained Ratio (%)":"30"}. This record does not assert independent reproduction.

### Qwen2.5-Omni-3B

VideoMME Performance: 62.51. Result ID: 6a06d88d47f34ce924ee7225.

[Reporting paper](<https://www.sota2.com/research/paper/6a06a876dbc836b87d829f24.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa77a21e209131cea7482c.md>)

Recorded evaluation configuration: {"Model Backbone":"Qwen2.5-Omni-3B","Retained Ratio (%)":"100"}. This record does not assert independent reproduction.

### DASH

VideoMME Performance: 62.26. Result ID: 6a06d88d47f34ce924ee7228.

[Reporting paper](<https://www.sota2.com/research/paper/6a06a876dbc836b87d829f24.md>)

[Method origin paper](<https://www.sota2.com/research/paper/69ba316d0636e5bf4afc0614.md>)

Recorded evaluation configuration: {"Model Backbone":"Qwen2.5-Omni-3B","Retained Ratio (%)":"30"}. This record does not assert independent reproduction.

### OmniZip

VideoMME Performance: 61.59. Result ID: 6a06d88d47f34ce924ee7227.

[Reporting paper](<https://www.sota2.com/research/paper/6a06a876dbc836b87d829f24.md>)

[Method origin paper](<https://www.sota2.com/research/paper/69a125012d603e6de3e3c43b.md>)

Recorded evaluation configuration: {"Model Backbone":"Qwen2.5-Omni-3B","Retained Ratio (%)":"20"}. This record does not assert independent reproduction.

### DASH

VideoMME Performance: 60.96. Result ID: 6a06d88d47f34ce924ee7229.

[Reporting paper](<https://www.sota2.com/research/paper/6a06a876dbc836b87d829f24.md>)

[Method origin paper](<https://www.sota2.com/research/paper/69ba316d0636e5bf4afc0614.md>)

Recorded evaluation configuration: {"Model Backbone":"Qwen2.5-Omni-3B","Retained Ratio (%)":"20"}. This record does not assert independent reproduction.

Page 1 of 1. This is an index page, not the full collection.
