# Multimodal Long-Context Understanding on MileBench

Source: [Multimodal Long-Context Understanding on MileBench](<https://www.sota2.com/research/sota/multimodal-long-context-understanding-on-milebench>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2026-04-21T08:26:47.126Z

Evaluation of multimodal models on the MileBench suite, focusing on long-context understanding tasks including Object Existence, Action Localization, DocVQA, TQA, SpotTheDiff, CLEVR-Change, and MMCoQA.

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

| Scope | Recorded value |
| --- | --- |
| Task | Multimodal Long-Context Understanding |
| Dataset (including any stated split) | MileBench |
| Metric | Object Existence |
| Unit | PERCENT |
| Direction | Higher is better |
| Results loaded | 8 of 8 |
| Coverage | All results currently returned for this benchmark |

Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol.

## Metrics

- [Object Existence](<https://www.sota2.com/research/sota/multimodal-long-context-understanding-on-milebench.md?metric=Object%20Existence>): Higher is better; unit: PERCENT.

- [Action Localization](<https://www.sota2.com/research/sota/multimodal-long-context-understanding-on-milebench.md?metric=Action%20Localization>): Higher is better; unit: PERCENT.

- [DocVQA](<https://www.sota2.com/research/sota/multimodal-long-context-understanding-on-milebench.md?metric=DocVQA>): Higher is better; unit: PERCENT.

- [TQA](<https://www.sota2.com/research/sota/multimodal-long-context-understanding-on-milebench.md?metric=TQA>): Higher is better; unit: PERCENT.

- [SpotTheDiff](<https://www.sota2.com/research/sota/multimodal-long-context-understanding-on-milebench.md?metric=SpotTheDiff>): Higher is better; unit: PERCENT.

- [CLEVR-Change](<https://www.sota2.com/research/sota/multimodal-long-context-understanding-on-milebench.md?metric=CLEVR-Change>): Higher is better; unit: PERCENT.

- [MMCoQA](<https://www.sota2.com/research/sota/multimodal-long-context-understanding-on-milebench.md?metric=MMCoQA>): Higher is better; unit: PERCENT.

- [Average Score (Avg.)](<https://www.sota2.com/research/sota/multimodal-long-context-understanding-on-milebench.md?metric=Average%20Score%20(Avg.)>): Higher is better; unit: PERCENT.

- [Average Score Change (Δ Avg.)](<https://www.sota2.com/research/sota/multimodal-long-context-understanding-on-milebench.md?metric=Average%20Score%20Change%20(%CE%94%20Avg.)>): Higher is better; unit: SCALAR.

## Recorded results

| Method | Object Existence | Result date | Report paper |
| --- | --- | --- | --- |
| Full Attention | 51.5 | Not recorded | 69e704767d0172140ec7ec01 |
| SinkRouter | 51.5 | Not recorded | 69e704767d0172140ec7ec01 |
| FastV | 51 | Not recorded | 69e704767d0172140ec7ec01 |
| LOOK-M | 50.5 | Not recorded | 69e704767d0172140ec7ec01 |
| FastV | 48.5 | Not recorded | 69e704767d0172140ec7ec01 |
| Full Attention | 47.5 | Not recorded | 69e704767d0172140ec7ec01 |
| LOOK-M | 47.5 | Not recorded | 69e704767d0172140ec7ec01 |
| SinkRouter | 47.5 | Not recorded | 69e704767d0172140ec7ec01 |

### Full Attention

Object Existence: 51.5. Result ID: 69e734c147f34ce924cd65a4.

[Reporting paper](<https://www.sota2.com/research/paper/69e704767d0172140ec7ec01.md>)

Recorded evaluation configuration: {"Backbone":"LLaVA-1.5-7B"}. This record does not assert independent reproduction.

### SinkRouter

Object Existence: 51.5. Result ID: 69e734c147f34ce924cd65a7.

[Reporting paper](<https://www.sota2.com/research/paper/69e704767d0172140ec7ec01.md>)

[Method origin paper](<https://www.sota2.com/research/paper/69e704767d0172140ec7ec01.md>)

Recorded evaluation configuration: {"Backbone":"LLaVA-1.5-7B"}. This record does not assert independent reproduction.

### FastV

Object Existence: 51. Result ID: 69e734c147f34ce924cd65a5.

[Reporting paper](<https://www.sota2.com/research/paper/69e704767d0172140ec7ec01.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa74ae1e209131cea632b7.md>)

Recorded evaluation configuration: {"Backbone":"LLaVA-1.5-7B"}. This record does not assert independent reproduction.

### LOOK-M

Object Existence: 50.5. Result ID: 69e734c147f34ce924cd65a6.

[Reporting paper](<https://www.sota2.com/research/paper/69e704767d0172140ec7ec01.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa73d31e209131cea5e5a8.md>)

Recorded evaluation configuration: {"Backbone":"LLaVA-1.5-7B"}. This record does not assert independent reproduction.

### FastV

Object Existence: 48.5. Result ID: 69e734c147f34ce924cd65a9.

[Reporting paper](<https://www.sota2.com/research/paper/69e704767d0172140ec7ec01.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa74ae1e209131cea632b7.md>)

Recorded evaluation configuration: {"Backbone":"LLaVA-1.5-13B"}. This record does not assert independent reproduction.

### Full Attention

Object Existence: 47.5. Result ID: 69e734c147f34ce924cd65a8.

[Reporting paper](<https://www.sota2.com/research/paper/69e704767d0172140ec7ec01.md>)

Recorded evaluation configuration: {"Backbone":"LLaVA-1.5-13B"}. This record does not assert independent reproduction.

### LOOK-M

Object Existence: 47.5. Result ID: 69e734c147f34ce924cd65aa.

[Reporting paper](<https://www.sota2.com/research/paper/69e704767d0172140ec7ec01.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa73d31e209131cea5e5a8.md>)

Recorded evaluation configuration: {"Backbone":"LLaVA-1.5-13B"}. This record does not assert independent reproduction.

### SinkRouter

Object Existence: 47.5. Result ID: 69e734c147f34ce924cd65ab.

[Reporting paper](<https://www.sota2.com/research/paper/69e704767d0172140ec7ec01.md>)

[Method origin paper](<https://www.sota2.com/research/paper/69e704767d0172140ec7ec01.md>)

Recorded evaluation configuration: {"Backbone":"LLaVA-1.5-13B"}. This record does not assert independent reproduction.

Page 1 of 1. This is an index page, not the full collection.