Latent Multi-hop Reasoning on HotpotQA
76.07PrecisionLlama 3.1-8B
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| Llama 3.1-8BPacking Strategy=Pack 22025.12 | 76.07 | 30.8 | 72.3 | |
| Llama 3.2-3BPacking Strategy=Pack 62025.12 | 71.86 | 24.85 | 67.52 | |
| Gemma 2-2BPacking Strategy=Pack 42025.12 | 71.8 | 13.4 | 64.63 | |
| Gemma 2-2BPacking Strategy=Pack 62025.12 | 71.36 | 20.65 | 62.85 | |
| Llama 3.2-3BPacking Strategy=Pack 2-42025.12 | 71.36 | 18.32 | 68.63 | |
| Llama 3.2-3BPacking Strategy=Pack 4-82025.12 | 71.13 | 26.51 | 65.63 | |
| Llama 3.2-3BPacking Strategy=Pack 42025.12 | 70.86 | 21.43 | 66.85 | |
| Gemma 2-2BPacking Strategy=Pack 22025.12 | 70.69 | 11.09 | 63.74 | |
| Llama 3.1-8BPacking Strategy=No packing2025.12 | 70.68 | 43.88 | 64.29 | |
| Llama 3.2-3BPacking Strategy=Pack 2-4-82025.12 | 70.52 | 25 | 65.52 | |
| Gemma 2-2BPacking Strategy=Pack 4-82025.12 | 70.13 | 25.66 | 62.74 | |
| Llama 3.2-3BPacking Strategy=Pack 22025.12 | 69.91 | 14.24 | 64.85 | |
| Gemma 2-2BPacking Strategy=Pack 82025.12 | 69.69 | 27.69 | 62.07 | |
| Gemma 2-2BPacking Strategy=Pack 2-4-82025.12 | 69.62 | 24.66 | 62.51 | |
| Llama 3.2-3BPacking Strategy=Pack 82025.12 | 69.58 | 28.38 | 66.18 | |
| Gemma 2-2BPacking Strategy=Pack 2-82025.12 | 69.56 | 27.68 | 63.18 | |
| Llama 3.2-3BPacking Strategy=Pack 2-82025.12 | 69.3 | 27.21 | 66.96 | |
| Gemma 2-2BPacking Strategy=Pack 2-42025.12 | 69.17 | 15.04 | 63.07 | |
| Llama 3.2-3BPacking Strategy=Pack 102025.12 | 68.63 | 44.49 | 62.63 | |
| Gemma 2-2BPacking Strategy=Pack 102025.12 | 68.41 | 47.4 | 57.73 | |
| Llama 3.2-3BPacking Strategy=No packing2025.12 | 66.17 | 31.24 | 56.73 | |
| Gemma 2-2BPacking Strategy=No packing2025.12 | 65.63 | 15.52 | 58.64 |