Text-based Question Answering on 2WikiMultihopQA (val) (EM/F1/R@k/#Tok)
24.3EMLatent Memory
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| Latent MemoryGeneration LLM=Meta-Llama-3-8B-Instruct, Retrieval k=52026.06 | 24.3 | 36.7 | 74.2 | 69 | |
| Latent MemoryGeneration LLM=Meta-Llama-3-8B-Instruct, Retrieval k=22026.06 | 21.5 | 33.5 | 49.5 | 42 | |
| Latent MemoryGeneration LLM=Meta-Llama-3-8B-Instruct, Retrieval k=12026.06 | 19.8 | 29.2 | 28.4 | 33 | |
| Dense RetrievalGeneration LLM=Meta-Llama-3-8B-Instruct, Retrieval k=52026.06 | 19.1 | 37.1 | 58 | 198 | |
| Qwen3-Emb-0.6BGeneration LLM=Meta-Llama-3-8B-Instruct, Retrieval k=52026.06 | 19.1 | 38.6 | 64.3 | 208 | |
| Dense RetrievalGeneration LLM=Meta-Llama-3-8B-Instruct, Retrieval k=12026.06 | 19 | 30 | 23.3 | 61 | |
| Dense RetrievalGeneration LLM=Meta-Llama-3-8B-Instruct, Retrieval k=22026.06 | 18.1 | 31.7 | 37.6 | 95 | |
| Qwen3-Emb-0.6BGeneration LLM=Meta-Llama-3-8B-Instruct, Retrieval k=12026.06 | 18.1 | 30.9 | 32.5 | 66 | |
| Full ContextGeneration LLM=Meta-Llama-3-8B-Instruct2026.06 | 17.7 | 39.2 | — | 1,074 | |
| Qwen3-Emb-0.6BGeneration LLM=Meta-Llama-3-8B-Instruct, Retrieval k=22026.06 | 17.5 | 33.5 | 47.7 | 102 | |
| BM25 RetrievalGeneration LLM=Meta-Llama-3-8B-Instruct, Retrieval k=12026.06 | 17.2 | 27.2 | 18.3 | 60 | |
| LLMLinguaGeneration LLM=Meta-Llama-3-8B-Instruct, Compression percentage=20%2026.06 | 17 | 30.4 | — | 199 | |
| BM25 RetrievalGeneration LLM=Meta-Llama-3-8B-Instruct, Retrieval k=22026.06 | 16.3 | 28.3 | 29.2 | 94 | |
| BM25 RetrievalGeneration LLM=Meta-Llama-3-8B-Instruct, Retrieval k=52026.06 | 16.3 | 32.7 | 46.9 | 196 | |
| LLMLinguaGeneration LLM=Meta-Llama-3-8B-Instruct, Compression percentage=5%2026.06 | 15 | 22.2 | — | 63 | |
| LLMLinguaGeneration LLM=Meta-Llama-3-8B-Instruct, Compression percentage=10%2026.06 | 14.8 | 24.7 | — | 108 |