Text-based Question Answering on HotpotQA (val)
42EMFull Context
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| Full ContextGeneration LLM=Meta-Llama-3-8B-Instruct2026.06 | 42 | 57.8 | — | 1,462 | |
| BM25 RetrievalGeneration LLM=Meta-Llama-3-8B-Instruct, Retrieval k=52026.06 | 41.3 | 55.3 | 65.5 | 224 | |
| Qwen3-Emb-0.6BGeneration LLM=Meta-Llama-3-8B-Instruct, Retrieval k=52026.06 | 40 | 54.5 | 70.1 | 224 | |
| Dense RetrievalGeneration LLM=Meta-Llama-3-8B-Instruct, Retrieval k=52026.06 | 37 | 50.8 | 62.4 | 214 | |
| BM25 RetrievalGeneration LLM=Meta-Llama-3-8B-Instruct, Retrieval k=22026.06 | 36.8 | 49.9 | 45.5 | 106 | |
| Qwen3-Emb-0.6BGeneration LLM=Meta-Llama-3-8B-Instruct, Retrieval k=22026.06 | 35.6 | 49 | 50.1 | 109 | |
| Latent MemoryGeneration LLM=Meta-Llama-3-8B-Instruct, Retrieval k=52026.06 | 34.8 | 48.9 | 87.1 | 72 | |
| Dense RetrievalGeneration LLM=Meta-Llama-3-8B-Instruct, Retrieval k=22026.06 | 32.6 | 45.4 | 42.2 | 104 | |
| BM25 RetrievalGeneration LLM=Meta-Llama-3-8B-Instruct, Retrieval k=12026.06 | 32.5 | 44.8 | 30.2 | 68 | |
| LLMLinguaGeneration LLM=Meta-Llama-3-8B-Instruct, Compression percentage=20%2026.06 | 31.8 | 44.8 | — | 283 | |
| Latent MemoryGeneration LLM=Meta-Llama-3-8B-Instruct, Retrieval k=22026.06 | 31.6 | 45.2 | 62.6 | 45 | |
| Qwen3-Emb-0.6BGeneration LLM=Meta-Llama-3-8B-Instruct, Retrieval k=12026.06 | 30.9 | 43.6 | 33.2 | 70 | |
| Dense RetrievalGeneration LLM=Meta-Llama-3-8B-Instruct, Retrieval k=12026.06 | 29.4 | 41 | 27.9 | 67 | |
| Latent MemoryGeneration LLM=Meta-Llama-3-8B-Instruct, Retrieval k=12026.06 | 27.4 | 39.4 | 34.6 | 36 | |
| LLMLinguaGeneration LLM=Meta-Llama-3-8B-Instruct, Compression percentage=10%2026.06 | 25 | 36.1 | — | 154 | |
| LLMLinguaGeneration LLM=Meta-Llama-3-8B-Instruct, Compression percentage=5%2026.06 | 20.9 | 30.2 | — | 87 |