Multimodal Question Answering on WebQA Text
54.9F1 ScoreFull Context
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| Full ContextGenerator=Gemma-3-12B-Instruct, Retrieval Method=Full-context reference2026.06 | 54.9 | 54.9 | — | 4,337 | |
| Qwen3-VL-8BGenerator=Gemma-3-12B-Instruct, Retrieval count (k)=52026.06 | 52.3 | 53.5 | 87.5 | 442 | |
| Nemo-EmbGenerator=Gemma-3-12B-Instruct, Retrieval count (k)=52026.06 | 52.1 | 53 | 87.1 | 450 | |
| BM25 RetrievalGenerator=Gemma-3-12B-Instruct, Retrieval count (k)=52026.06 | 49.5 | 50.3 | 73 | 415 | |
| Nemo-EmbGenerator=Gemma-3-12B-Instruct, Retrieval count (k)=22026.06 | 48.9 | 50.1 | 66.6 | 186 | |
| Qwen3-VL-8BGenerator=Gemma-3-12B-Instruct, Retrieval count (k)=22026.06 | 47.6 | 48.7 | 66.3 | 187 | |
| Dense RetrievalGenerator=Gemma-3-12B-Instruct, Retrieval count (k)=52026.06 | 45.2 | 45.9 | 62 | 750 | |
| BM25 RetrievalGenerator=Gemma-3-12B-Instruct, Retrieval count (k)=22026.06 | 43.6 | 44.8 | 51 | 183 | |
| Nemo-EmbGenerator=Gemma-3-12B-Instruct, Retrieval count (k)=12026.06 | 41.5 | 43.3 | 40.5 | 107 | |
| Qwen3-VL-8BGenerator=Gemma-3-12B-Instruct, Retrieval count (k)=12026.06 | 40.7 | 42.5 | 40.8 | 108 | |
| Dense RetrievalGenerator=Gemma-3-12B-Instruct, Retrieval count (k)=22026.06 | 39 | 40 | 40.1 | 303 | |
| BM25 RetrievalGenerator=Gemma-3-12B-Instruct, Retrieval count (k)=12026.06 | 38.6 | 40.4 | 31.8 | 106 | |
| Dense RetrievalGenerator=Gemma-3-12B-Instruct, Retrieval count (k)=12026.06 | 36.2 | 37 | 25.5 | 158 | |
| Latent MemoryGenerator=Gemma-3-12B-Instruct, Retrieval count (k)=22026.06 | 31.9 | 32.3 | 41.3 | 48 | |
| Latent MemoryGenerator=Gemma-3-12B-Instruct, Retrieval count (k)=12026.06 | 31.6 | 31.9 | 23.7 | 39 | |
| Latent MemoryGenerator=Gemma-3-12B-Instruct, Retrieval count (k)=52026.06 | 31.3 | 31.7 | 70.5 | 75 |