# Multimodal Autocomplete on MMDD unseen prefixes

Source: [Multimodal Autocomplete on MMDD unseen prefixes](<https://www.sota2.com/research/sota/multimodal-autocomplete-on-mmdd-unseen-prefixes>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2026-02-26T05:55:16.392Z

Evaluation of Multimodal Autocomplete performance on the MMDD unseen prefixes dataset, using metrics like PR-F1, SM, and inference time.

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

| Scope | Recorded value |
| --- | --- |
| Task | Multimodal Autocomplete |
| Dataset (including any stated split) | MMDD unseen prefixes |
| Metric | PR-F1 |
| Unit | PERCENT |
| Direction | Higher is better |
| Results loaded | 15 of 15 |
| Coverage | All results currently returned for this benchmark |

Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol.

## Metrics

- [PR-F1](<https://www.sota2.com/research/sota/multimodal-autocomplete-on-mmdd-unseen-prefixes.md?metric=PR-F1>): Higher is better; unit: PERCENT.

- [SM](<https://www.sota2.com/research/sota/multimodal-autocomplete-on-mmdd-unseen-prefixes.md?metric=SM>): Higher is better; unit: PERCENT.

- [Time (s)](<https://www.sota2.com/research/sota/multimodal-autocomplete-on-mmdd-unseen-prefixes.md?metric=Time%20(s)>): Lower is better; unit: TIME.

- [TR](<https://www.sota2.com/research/sota/multimodal-autocomplete-on-mmdd-unseen-prefixes.md?metric=TR>): Higher is better; unit: PERCENT.

- [PR-P](<https://www.sota2.com/research/sota/multimodal-autocomplete-on-mmdd-unseen-prefixes.md?metric=PR-P>): Higher is better; unit: PERCENT.

- [PR-R](<https://www.sota2.com/research/sota/multimodal-autocomplete-on-mmdd-unseen-prefixes.md?metric=PR-R>): Higher is better; unit: PERCENT.

- [Avg Pred Length](<https://www.sota2.com/research/sota/multimodal-autocomplete-on-mmdd-unseen-prefixes.md?metric=Avg%20Pred%20Length>): Higher is better; unit: SCALAR.

- [TES](<https://www.sota2.com/research/sota/multimodal-autocomplete-on-mmdd-unseen-prefixes.md?metric=TES>): Lower is better; unit: SCALAR.

## Recorded results

| Method | PR-F1 | Result date | Report paper |
| --- | --- | --- | --- |
| Router-4-Max (Oracle) | 35.6 | Not recorded | 69647fda53fdd5c3617445a9 |
| Router-4-P | 28.1 | Not recorded | 69647fda53fdd5c3617445a9 |
| MiniCPM-V | 28 | Not recorded | 69647fda53fdd5c3617445a9 |
| Router-2-P | 26.1 | Not recorded | 69647fda53fdd5c3617445a9 |
| Qwen2-VL | 25.32 | Not recorded | 69647fda53fdd5c3617445a9 |
| MiniCPM-V | 24.7 | Not recorded | 69647fda53fdd5c3617445a9 |
| PaliGemma | 24.7 | Not recorded | 69647fda53fdd5c3617445a9 |
| Router-4-L | 24 | Not recorded | 69647fda53fdd5c3617445a9 |
| Router-2-L | 24 | Not recorded | 69647fda53fdd5c3617445a9 |
| Qwen2-VL | 22.2 | Not recorded | 69647fda53fdd5c3617445a9 |
| PaliGemma | 21.6 | Not recorded | 69647fda53fdd5c3617445a9 |
| QB | 20.9 | Not recorded | 69647fda53fdd5c3617445a9 |
| QB | 18.92 | Not recorded | 69647fda53fdd5c3617445a9 |
| MPC++ | 15.25 | Not recorded | 69647fda53fdd5c3617445a9 |
| MPC | 7.25 | Not recorded | 69647fda53fdd5c3617445a9 |

### Router-4-Max (Oracle)

PR-F1: 35.6. Result ID: 699693995698665ec4dcd9f6.

[Reporting paper](<https://www.sota2.com/research/paper/69647fda53fdd5c3617445a9.md>)

[Method origin paper](<https://www.sota2.com/research/paper/69647fda53fdd5c3617445a9.md>)

[Implementation](<https://github.com/devichand579/MAC>)

Recorded evaluation configuration: {"Router configuration":"Router-4","Oracle":true}. This record does not assert independent reproduction.

### Router-4-P

PR-F1: 28.1. Result ID: 699693995698665ec4dcd9f3.

[Reporting paper](<https://www.sota2.com/research/paper/69647fda53fdd5c3617445a9.md>)

[Method origin paper](<https://www.sota2.com/research/paper/69647fda53fdd5c3617445a9.md>)

[Implementation](<https://github.com/devichand579/MAC>)

Recorded evaluation configuration: {"Router configuration":"Router-4","Optimization goal":"P (Maximum performance)","Inference engine":"vLLM"}. This record does not assert independent reproduction.

### MiniCPM-V

PR-F1: 28. Result ID: 699693995698665ec4dcda03.

[Reporting paper](<https://www.sota2.com/research/paper/69647fda53fdd5c3617445a9.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa74681e209131cea6183d.md>)

[Implementation](<https://github.com/OpenBMB/MiniCPM-V>)

Recorded evaluation configuration: {"Type":"VLMs"}. This record does not assert independent reproduction.

### Router-2-P

PR-F1: 26.1. Result ID: 699693995698665ec4dcd9f5.

[Reporting paper](<https://www.sota2.com/research/paper/69647fda53fdd5c3617445a9.md>)

[Method origin paper](<https://www.sota2.com/research/paper/69647fda53fdd5c3617445a9.md>)

[Implementation](<https://github.com/devichand579/MAC>)

Recorded evaluation configuration: {"Router configuration":"Router-2","Optimization goal":"P (Maximum performance)","Inference engine":"vLLM"}. This record does not assert independent reproduction.

### Qwen2-VL

PR-F1: 25.32. Result ID: 699693995698665ec4dcda05.

[Reporting paper](<https://www.sota2.com/research/paper/69647fda53fdd5c3617445a9.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa75151e209131cea65976.md>)

[Implementation](<https://github.com/QwenLM/Qwen2-VL>)

Recorded evaluation configuration: {"Type":"VLMs"}. This record does not assert independent reproduction.

### MiniCPM-V

PR-F1: 24.7. Result ID: 699693995698665ec4dcd9ee.

[Reporting paper](<https://www.sota2.com/research/paper/69647fda53fdd5c3617445a9.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa74681e209131cea6183d.md>)

[Implementation](<https://github.com/OpenBMB/MiniCPM-V>)

Recorded evaluation configuration: {"Model":"Individual Model","Inference engine":"vLLM"}. This record does not assert independent reproduction.

### PaliGemma

PR-F1: 24.7. Result ID: 699693995698665ec4dcda04.

[Reporting paper](<https://www.sota2.com/research/paper/69647fda53fdd5c3617445a9.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa751e1e209131cea65e0c.md>)

Recorded evaluation configuration: {"Type":"VLMs"}. This record does not assert independent reproduction.

### Router-4-L

PR-F1: 24. Result ID: 699693995698665ec4dcd9f2.

[Reporting paper](<https://www.sota2.com/research/paper/69647fda53fdd5c3617445a9.md>)

[Method origin paper](<https://www.sota2.com/research/paper/69647fda53fdd5c3617445a9.md>)

[Implementation](<https://github.com/devichand579/MAC>)

Recorded evaluation configuration: {"Router configuration":"Router-4","Optimization goal":"L (Minimum latency)","Inference engine":"vLLM"}. This record does not assert independent reproduction.

### Router-2-L

PR-F1: 24. Result ID: 699693995698665ec4dcd9f4.

[Reporting paper](<https://www.sota2.com/research/paper/69647fda53fdd5c3617445a9.md>)

[Method origin paper](<https://www.sota2.com/research/paper/69647fda53fdd5c3617445a9.md>)

[Implementation](<https://github.com/devichand579/MAC>)

Recorded evaluation configuration: {"Router configuration":"Router-2","Optimization goal":"L (Minimum latency)","Inference engine":"vLLM"}. This record does not assert independent reproduction.

### Qwen2-VL

PR-F1: 22.2. Result ID: 699693995698665ec4dcd9f1.

[Reporting paper](<https://www.sota2.com/research/paper/69647fda53fdd5c3617445a9.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa75151e209131cea65976.md>)

[Implementation](<https://github.com/QwenLM/Qwen2-VL>)

Recorded evaluation configuration: {"Model":"Individual Model","Inference engine":"vLLM"}. This record does not assert independent reproduction.

### PaliGemma

PR-F1: 21.6. Result ID: 699693995698665ec4dcd9ef.

[Reporting paper](<https://www.sota2.com/research/paper/69647fda53fdd5c3617445a9.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa751e1e209131cea65e0c.md>)

Recorded evaluation configuration: {"Model":"Individual Model","Inference engine":"vLLM"}. This record does not assert independent reproduction.

### QB

PR-F1: 20.9. Result ID: 699693995698665ec4dcd9f0.

[Reporting paper](<https://www.sota2.com/research/paper/69647fda53fdd5c3617445a9.md>)

Recorded evaluation configuration: {"Model":"Individual Model","Inference engine":"vLLM"}. This record does not assert independent reproduction.

### QB

PR-F1: 18.92. Result ID: 699693995698665ec4dcda02.

[Reporting paper](<https://www.sota2.com/research/paper/69647fda53fdd5c3617445a9.md>)

Recorded evaluation configuration: {"Type":"Textual"}. This record does not assert independent reproduction.

### MPC++

PR-F1: 15.25. Result ID: 699693995698665ec4dcda01.

[Reporting paper](<https://www.sota2.com/research/paper/69647fda53fdd5c3617445a9.md>)

Recorded evaluation configuration: {"Type":"Textual"}. This record does not assert independent reproduction.

### MPC

PR-F1: 7.25. Result ID: 699693995698665ec4dcda00.

[Reporting paper](<https://www.sota2.com/research/paper/69647fda53fdd5c3617445a9.md>)

Recorded evaluation configuration: {"Type":"Textual"}. This record does not assert independent reproduction.

Page 1 of 1. This is an index page, not the full collection.
