Multi-hop Question Answering on HotpotQA (EM, F1, Rate)
55Exact Match (EM)BRIEF-PRO-HIGH
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| BRIEF-PRO-HIGHCategory=Abstractive, Reader Model (M)=Llama-3.1-70B-Instruct2025.10 | 55 | 67.42 | 72 | |
| BRIEF-PRO-MEDIUMCategory=Abstractive, Reader Model (M)=Llama-3.1-70B-Instruct2025.10 | 54.5 | 68.02 | 42 | |
| BRIEF-PRO-LOWCategory=Abstractive, Reader Model (M)=Llama-3.1-70B-Instruct2025.10 | 54.5 | 68.35 | 26 | |
| BRIEF-PRO-AUTOCategory=Abstractive, Reader Model (M)=Llama-3.1-70B-Instruct2025.10 | 53 | 67.29 | 34 | |
| BRIEF-PRO-MEDIUMCategory=Abstractive, Reader Model (M)=GPT-4.1-nano2025.10 | 52 | 65.38 | 42 | |
| Non-compressionCategory=Non-compression, Reader Model (M)=Llama-3.1-70B-Instruct2025.10 | 51.5 | 65.14 | 1 | |
| BRIEF-PRO-AUTOCategory=Abstractive, Reader Model (M)=GPT-4.1-nano2025.10 | 51.5 | 65.52 | 34 | |
| BRIEF-PRO-MEDIUMCategory=Abstractive, Reader Model (M)=Llama-3.1-8B-Instruct2025.10 | 51 | 63.41 | 42 | |
| BRIEF-PRO-HIGHCategory=Abstractive, Reader Model (M)=GPT-4.1-nano2025.10 | 51 | 64.13 | 72 | |
| BRIEF-PRO-LOWCategory=Abstractive, Reader Model (M)=GPT-4.1-nano2025.10 | 51 | 64.86 | 26 | |
| BRIEF-PRO-LOWCategory=Abstractive, Reader Model (M)=Llama-3.1-8B-Instruct2025.10 | 50.5 | 64.36 | 26 | |
| Rerank Top-5Category=Extractive, Reader Model (M)=Llama-3.1-70B-Instruct2025.10 | 50 | 64.45 | 1.5 | |
| BRIEF-PRO-AUTOCategory=Abstractive, Reader Model (M)=Llama-3.1-8B-Instruct2025.10 | 49 | 63.05 | 34 | |
| FILM-7BCategory=Long-context LLMs, Reader Model (M)=FILM-7B2025.10 | 47.5 | 61.43 | 1 | |
| BRIEF-PRO-AUTO_L7CCategory=Abstractive, Reader Model (M)=Llama-3.1-70B-Instruct2025.10 | 47.5 | 60.97 | 13 | |
| BRIEF-PRO-HIGHCategory=Abstractive, Reader Model (M)=Llama-3.1-8B-Instruct2025.10 | 47 | 58.85 | 72 | |
| EXITCategory=Extractive, Reader Model (M)=Llama-3.1-70B-Instruct2025.10 | 46 | 59.8 | 32 | |
| LongLLMLinguaCategory=Abstractive, Reader Model (M)=Llama-3.1-70B-Instruct2025.10 | 46 | 61.32 | 8 | |
| BRIEF-PRO-AUTO_L7CCategory=Abstractive, Reader Model (M)=GPT-4.1-nano2025.10 | 46 | 56.06 | 13 | |
| Rerank Top-3Category=Extractive, Reader Model (M)=Llama-3.1-70B-Instruct2025.10 | 43 | 55.78 | 2.2 | |
| ProLong-8BCategory=Long-context LLMs, Reader Model (M)=ProLong-8B2025.10 | 42.5 | 54.43 | 1 | |
| BRIEF-PRO-AUTO_L7CCategory=Abstractive, Reader Model (M)=Llama-3.1-8B-Instruct2025.10 | 42.5 | 56.17 | 13 | |
| Non-compressionCategory=Non-compression, Reader Model (M)=GPT-4.1-nano2025.10 | 42.5 | 56.56 | 1 | |
| Rerank Top-5Category=Extractive, Reader Model (M)=Llama-3.1-8B-Instruct2025.10 | 42 | 55.16 | 1.5 | |
| GPT-4.1-nanoCategory=Abstractive, Reader Model (M)=Llama-3.1-70B-Instruct2025.10 | 42 | 56.59 | 90 | |
| EXITCategory=Extractive, Reader Model (M)=GPT-4.1-nano2025.10 | 42 | 54.41 | 32 | |
| RECOMP (Extractive)Category=Extractive, Reader Model (M)=GPT-4.1-nano2025.10 | 41 | 52.44 | 36 | |
| Rerank Top-5Category=Extractive, Reader Model (M)=GPT-4.1-nano2025.10 | 41 | 54.87 | 1.5 | |
| LongLLMLinguaCategory=Abstractive, Reader Model (M)=GPT-4.1-nano2025.10 | 40.5 | 54.9 | 8 | |
| GPT-4.1-nanoCategory=Abstractive, Reader Model (M)=GPT-4.1-nano2025.10 | 40.5 | 55.87 | 90 | |
| Non-compressionCategory=Non-compression, Reader Model (M)=Llama-3.1-8B-Instruct2025.10 | 40 | 54.29 | 1 | |
| RECOMP (Extractive)Category=Extractive, Reader Model (M)=Llama-3.1-70B-Instruct2025.10 | 40 | 53.27 | 36 | |
| Rerank Top-3Category=Extractive, Reader Model (M)=GPT-4.1-nano2025.10 | 40 | 52.85 | 2.2 | |
| LongLLMLinguaCategory=Abstractive, Reader Model (M)=Llama-3.1-8B-Instruct2025.10 | 38.5 | 53.63 | 8 | |
| GPT-4.1-nanoCategory=Abstractive, Reader Model (M)=Llama-3.1-8B-Instruct2025.10 | 38 | 53.67 | 90 | |
| Rerank Top-3Category=Extractive, Reader Model (M)=Llama-3.1-8B-Instruct2025.10 | 37.5 | 50.07 | 2.2 | |
| RECOMP (Abstractive)Category=Abstractive, Reader Model (M)=Llama-3.1-70B-Instruct2025.10 | 37.5 | 50.47 | 10 | |
| EXITCategory=Extractive, Reader Model (M)=Llama-3.1-8B-Instruct2025.10 | 36.5 | 48.62 | 32 | |
| RECOMP (Extractive)Category=Extractive, Reader Model (M)=Llama-3.1-8B-Instruct2025.10 | 35 | 48.48 | 36 | |
| Llama-3.2-3B-InstructCategory=Abstractive, Reader Model (M)=Llama-3.1-70B-Instruct2025.10 | 34.5 | 47.26 | 41 | |
| Llama-3.2-3B-InstructCategory=Abstractive, Reader Model (M)=GPT-4.1-nano2025.10 | 33 | 44.72 | 41 | |
| Rerank Top-1Category=Extractive, Reader Model (M)=Llama-3.1-70B-Instruct2025.10 | 32 | 44.42 | 8 | |
| Llama-3.2-3B-InstructCategory=Abstractive, Reader Model (M)=Llama-3.1-8B-Instruct2025.10 | 30 | 41.67 | 41 | |
| Rerank Top-1Category=Extractive, Reader Model (M)=GPT-4.1-nano2025.10 | 30 | 40.85 | 8 | |
| RECOMP (Abstractive)Category=Abstractive, Reader Model (M)=Llama-3.1-8B-Instruct2025.10 | 28.5 | 40.59 | 10 | |
| RECOMP (Abstractive)Category=Abstractive, Reader Model (M)=GPT-4.1-nano2025.10 | 27 | 39.21 | 10 | |
| BRIEFCategory=Abstractive, Reader Model (M)=GPT-4.1-nano2025.10 | 25.5 | 34.75 | 59 | |
| BRIEFCategory=Abstractive, Reader Model (M)=Llama-3.1-70B-Instruct2025.10 | 25 | 36.03 | 59 | |
| Rerank Top-1Category=Extractive, Reader Model (M)=Llama-3.1-8B-Instruct2025.10 | 23.5 | 35.66 | 8 | |
| BRIEFCategory=Abstractive, Reader Model (M)=Llama-3.1-8B-Instruct2025.10 | 18 | 27.7 | 59 |