Multimodal Question Answering on WebQA Average
50.8F1 ScoreNemo-Emb-1B
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| Nemo-Emb-1Bk=5, Generator VLM=LLaVA-1.5-13B, #Tok=12572026.06 | 50.8 | 47.6 | 84.3 | — | |
| Latent Memoryk=5, Generator VLM=LLaVA-1.5-13B, #Tok=832026.06 | 50 | 39.2 | 80.8 | — | |
| Dense Retrievalk=5, Generator VLM=LLaVA-1.5-13B, #Tok=15202026.06 | 43.7 | 48.1 | 64.6 | — | |
| Latent Memoryk=2, Generator VLM=LLaVA-1.5-13B, #Tok=532026.06 | 43.2 | 36.2 | 57.9 | — | |
| Qwen3-VL-Emb-8Bk=5, Generator VLM=LLaVA-1.5-13B, #Tok=7842026.06 | 41 | 43.8 | 68.5 | — | |
| Nemo-Emb-1Bk=2, Generator VLM=LLaVA-1.5-13B, #Tok=5632026.06 | 39.9 | 40.8 | 65.7 | — | |
| BM25 Retrievalk=5, Generator VLM=LLaVA-1.5-13B, #Tok=7422026.06 | 39.6 | 41.9 | 55.5 | — | |
| Full ContextGenerator=Gemma-3-12B-Instruct, Retrieval Method=Full-context reference2026.06 | 34.3 | 36.4 | — | 5,097 | |
| Dense Retrievalk=2, Generator VLM=LLaVA-1.5-13B, #Tok=6762026.06 | 34.2 | 39.7 | 46.6 | — | |
| Qwen3-VL-Emb-8Bk=2, Generator VLM=LLaVA-1.5-13B, #Tok=3502026.06 | 33.1 | 37.3 | 49.6 | — | |
| Nemo-EmbGenerator=Gemma-3-12B-Instruct, Retrieval count (k)=52026.06 | 31.7 | 35.1 | 84.3 | 698 | |
| BM25 Retrievalk=2, Generator VLM=LLaVA-1.5-13B, #Tok=3552026.06 | 31.6 | 35.9 | 39.6 | — | |
| Qwen3-VL-8BGenerator=Gemma-3-12B-Instruct, Retrieval count (k)=52026.06 | 30.9 | 32.9 | 68.5 | 506 | |
| Nemo-Emb-1Bk=1, Generator VLM=LLaVA-1.5-13B, #Tok=3192026.06 | 30.5 | 34.6 | 44.6 | — | |
| Latent Memoryk=1, Generator VLM=LLaVA-1.5-13B, #Tok=432026.06 | 30.4 | 29.7 | 40.3 | — | |
| Nemo-EmbGenerator=Gemma-3-12B-Instruct, Retrieval count (k)=22026.06 | 30 | 33.5 | 65.7 | 315 | |
| BM25 RetrievalGenerator=Gemma-3-12B-Instruct, Retrieval count (k)=52026.06 | 29.2 | 30.4 | 55.5 | 487 | |
| Qwen3-VL-8BGenerator=Gemma-3-12B-Instruct, Retrieval count (k)=22026.06 | 28.4 | 29.9 | 49.5 | 228 | |
| Qwen3-VL-Emb-8Bk=1, Generator VLM=LLaVA-1.5-13B, #Tok=2082026.06 | 28 | 32.8 | 32.4 | — | |
| Dense RetrievalGenerator=Gemma-3-12B-Instruct, Retrieval count (k)=52026.06 | 27.9 | 30.2 | 64.5 | 799 | |
| BM25 Retrievalk=1, Generator VLM=LLaVA-1.5-13B, #Tok=2142026.06 | 27.2 | 32.1 | 26.8 | — | |
| Dense Retrievalk=1, Generator VLM=LLaVA-1.5-13B, #Tok=3702026.06 | 26.9 | 32.4 | 32.6 | — | |
| Nemo-EmbGenerator=Gemma-3-12B-Instruct, Retrieval count (k)=12026.06 | 26.4 | 30.5 | 44.6 | 181 | |
| BM25 RetrievalGenerator=Gemma-3-12B-Instruct, Retrieval count (k)=22026.06 | 26 | 27.2 | 39.6 | 228 | |
| Qwen3-VL-8BGenerator=Gemma-3-12B-Instruct, Retrieval count (k)=12026.06 | 24.7 | 27.1 | 32.4 | 136 | |
| Dense RetrievalGenerator=Gemma-3-12B-Instruct, Retrieval count (k)=22026.06 | 24.6 | 27.3 | 46.6 | 357 | |
| BM25 RetrievalGenerator=Gemma-3-12B-Instruct, Retrieval count (k)=12026.06 | 23.4 | 24.8 | 26.8 | 137 | |
| Dense RetrievalGenerator=Gemma-3-12B-Instruct, Retrieval count (k)=12026.06 | 23.4 | 26.4 | 32.6 | 200 | |
| Latent MemoryGenerator=Gemma-3-12B-Instruct, Retrieval count (k)=12026.06 | 21.6 | 25.3 | 38.1 | 38 | |
| Latent MemoryGenerator=Gemma-3-12B-Instruct, Retrieval count (k)=22026.06 | 21.6 | 25.1 | 55.7 | 48 | |
| Latent MemoryGenerator=Gemma-3-12B-Instruct, Retrieval count (k)=52026.06 | 21.3 | 24.6 | 79.8 | 74 | |
| Full ContextGenerator VLM=LLaVA-1.5-13B, #Tok=100132026.06 | 3 | 5 | — | — |