Multimodal Autocomplete on MMDD unseen prefixes
35.6PR-F1Router-4-Max (Oracle)
Evaluation Results
| Method | Links | ||||||||
|---|---|---|---|---|---|---|---|---|---|
| Router-4-Max (Oracle)Router configuration=Router-4, Oracle=true2026.01 | 35.6 | 19.5 | — | — | — | — | — | — | |
| Router-4-PRouter configuration=Router-4, Optimization goal=P (Maximum performance), Inference engine=vLLM2026.01 | 28.1 | 13.5 | 0.832 | — | — | — | — | — | |
| MiniCPM-VType=VLMs2026.01 | 28 | 11.82 | — | 98.98 | 33.62 | 24.23 | 21.1 | 0.2136 | |
| Router-2-PRouter configuration=Router-2, Optimization goal=P (Maximum performance), Inference engine=vLLM2026.01 | 26.1 | 12.2 | 0.271 | — | — | — | — | — | |
| Qwen2-VLType=VLMs2026.01 | 25.32 | 10.34 | — | 98.91 | 29.5 | 22.23 | 18.8 | 0.1844 | |
| MiniCPM-VModel=Individual Model, Inference engine=vLLM2026.01 | 24.7 | 11.6 | 2.08 | — | — | — | — | — | |
| PaliGemmaType=VLMs2026.01 | 24.7 | 9.72 | — | 98.8 | 28.96 | 21.45 | 20.3 | 0.203 | |
| Router-4-LRouter configuration=Router-4, Optimization goal=L (Minimum latency), Inference engine=vLLM2026.01 | 24 | 11 | 0.351 | — | — | — | — | — | |
| Router-2-LRouter configuration=Router-2, Optimization goal=L (Minimum latency), Inference engine=vLLM2026.01 | 24 | 10.9 | 0.17 | — | — | — | — | — | |
| Qwen2-VLModel=Individual Model, Inference engine=vLLM2026.01 | 22.2 | 10.1 | 0.733 | — | — | — | — | — | |
| PaliGemmaModel=Individual Model, Inference engine=vLLM2026.01 | 21.6 | 9.7 | 1.49 | — | — | — | — | — | |
| QBModel=Individual Model, Inference engine=vLLM2026.01 | 20.9 | 10.2 | 0.001 | — | — | — | — | — | |
| QBType=Textual2026.01 | 18.92 | 4.26 | — | 92.2 | 34.98 | 12.87 | 8.9 | 0.1724 | |
| MPC++Type=Textual2026.01 | 15.25 | 3.32 | — | 56.51 | 18.31 | 13.03 | 29.4 | 0.043 | |
| MPCType=Textual2026.01 | 7.25 | 0 | — | 19.91 | 7.82 | 6.76 | 40.6 | 0.0015 |