Multi-hop Question Answering on Average (MuSiQue, HotpotQA, 2WikiMultiHopQA, LongSeal)
46.49Average QA ScoreBRIEF-PRO-LOW
Evaluation results
| Method | Links | ||
|---|---|---|---|
| BRIEF-PRO-LOWCategory=Abstractive, Reader Model (M)=Llama-3.1-70B-Instruct2025.10 | 46.49 | 25 | |
| BRIEF-PRO-AUTOCategory=Abstractive, Reader Model (M)=Llama-3.1-70B-Instruct2025.10 | 45.58 | 32 | |
| BRIEF-PRO-MEDIUMCategory=Abstractive, Reader Model (M)=Llama-3.1-70B-Instruct2025.10 | 45.33 | 42 | |
| Non-compressionCategory=Non-compression, Reader Model (M)=Llama-3.1-70B-Instruct2025.10 | 44.98 | 1 | |
| BRIEF-PRO-HIGHCategory=Abstractive, Reader Model (M)=Llama-3.1-70B-Instruct2025.10 | 43.01 | 68 | |
| Rerank Top-5Category=Extractive, Reader Model (M)=Llama-3.1-70B-Instruct2025.10 | 41.55 | 2 | |
| BRIEF-PRO-LOWCategory=Abstractive, Reader Model (M)=GPT-4.1-nano2025.10 | 41.4 | 25 | |
| BRIEF-PRO-MEDIUMCategory=Abstractive, Reader Model (M)=GPT-4.1-nano2025.10 | 41.37 | 42 | |
| BRIEF-PRO-AUTO_L7CCategory=Abstractive, Reader Model (M)=Llama-3.1-70B-Instruct2025.10 | 41.22 | 12 | |
| LongLLMLinguaCategory=Abstractive, Reader Model (M)=Llama-3.1-70B-Instruct2025.10 | 40.91 | 9 | |
| BRIEF-PRO-AUTOCategory=Abstractive, Reader Model (M)=GPT-4.1-nano2025.10 | 40.8 | 32 | |
| BRIEF-PRO-LOWCategory=Abstractive, Reader Model (M)=Llama-3.1-8B-Instruct2025.10 | 40.06 | 25 | |
| BRIEF-PRO-HIGHCategory=Abstractive, Reader Model (M)=GPT-4.1-nano2025.10 | 39.59 | 68 | |
| GPT-4.1-nanoCategory=Abstractive, Reader Model (M)=Llama-3.1-70B-Instruct2025.10 | 39.28 | 110 | |
| GPT-4.1-nanoCategory=Abstractive, Reader Model (M)=GPT-4.1-nano2025.10 | 38.99 | 110 | |
| BRIEF-PRO-AUTOCategory=Abstractive, Reader Model (M)=Llama-3.1-8B-Instruct2025.10 | 38.79 | 32 | |
| BRIEF-PRO-MEDIUMCategory=Abstractive, Reader Model (M)=Llama-3.1-8B-Instruct2025.10 | 38.64 | 42 | |
| GPT-4.1-nanoCategory=Abstractive, Reader Model (M)=Llama-3.1-8B-Instruct2025.10 | 36.87 | 110 | |
| Rerank Top-3Category=Extractive, Reader Model (M)=Llama-3.1-70B-Instruct2025.10 | 36.52 | 3 | |
| EXITCategory=Extractive, Reader Model (M)=Llama-3.1-70B-Instruct2025.10 | 36.21 | 26 | |
| FILM-7BCategory=Long-context LLMs, Reader Model (M)=FILM-7B2025.10 | 35.18 | 1 | |
| BRIEF-PRO-AUTO_L7CCategory=Abstractive, Reader Model (M)=GPT-4.1-nano2025.10 | 34.08 | 12 | |
| BRIEF-PRO-HIGHCategory=Abstractive, Reader Model (M)=Llama-3.1-8B-Instruct2025.10 | 34.02 | 68 | |
| Rerank Top-5Category=Extractive, Reader Model (M)=GPT-4.1-nano2025.10 | 33.76 | 2 | |
| Non-compressionCategory=Non-compression, Reader Model (M)=GPT-4.1-nano2025.10 | 33.53 | 1 | |
| BRIEF-PRO-AUTO_L7CCategory=Abstractive, Reader Model (M)=Llama-3.1-8B-Instruct2025.10 | 33.11 | 12 | |
| LongLLMLinguaCategory=Abstractive, Reader Model (M)=GPT-4.1-nano2025.10 | 33.03 | 9 | |
| Rerank Top-3Category=Extractive, Reader Model (M)=GPT-4.1-nano2025.10 | 33.01 | 3 | |
| RECOMP (Extractive)Category=Extractive, Reader Model (M)=Llama-3.1-70B-Instruct2025.10 | 32.97 | 33 | |
| Rerank Top-5Category=Extractive, Reader Model (M)=Llama-3.1-8B-Instruct2025.10 | 32.86 | 2 | |
| Non-compressionCategory=Non-compression, Reader Model (M)=Llama-3.1-8B-Instruct2025.10 | 32.09 | 1 | |
| LongLLMLinguaCategory=Abstractive, Reader Model (M)=Llama-3.1-8B-Instruct2025.10 | 32.02 | 9 | |
| Llama-3.2-3B-InstructCategory=Abstractive, Reader Model (M)=Llama-3.1-70B-Instruct2025.10 | 31.37 | 47 | |
| EXITCategory=Extractive, Reader Model (M)=GPT-4.1-nano2025.10 | 30.65 | 26 | |
| ProLong-8BCategory=Long-context LLMs, Reader Model (M)=ProLong-8B2025.10 | 30.2 | 1 | |
| RECOMP (Abstractive)Category=Abstractive, Reader Model (M)=Llama-3.1-70B-Instruct2025.10 | 30.13 | 9.5 | |
| RECOMP (Extractive)Category=Extractive, Reader Model (M)=GPT-4.1-nano2025.10 | 29.21 | 33 | |
| Rerank Top-3Category=Extractive, Reader Model (M)=Llama-3.1-8B-Instruct2025.10 | 28.65 | 3 | |
| Llama-3.2-3B-InstructCategory=Abstractive, Reader Model (M)=GPT-4.1-nano2025.10 | 28.06 | 47 | |
| EXITCategory=Extractive, Reader Model (M)=Llama-3.1-8B-Instruct2025.10 | 26.06 | 26 | |
| Llama-3.2-3B-InstructCategory=Abstractive, Reader Model (M)=Llama-3.1-8B-Instruct2025.10 | 25.06 | 47 | |
| RECOMP (Extractive)Category=Extractive, Reader Model (M)=Llama-3.1-8B-Instruct2025.10 | 24.83 | 33 | |
| Rerank Top-1Category=Extractive, Reader Model (M)=Llama-3.1-70B-Instruct2025.10 | 23.25 | 10 | |
| BRIEFCategory=Abstractive, Reader Model (M)=Llama-3.1-70B-Instruct2025.10 | 22.13 | 32 | |
| RECOMP (Abstractive)Category=Abstractive, Reader Model (M)=Llama-3.1-8B-Instruct2025.10 | 21.75 | 10 | |
| BRIEFCategory=Abstractive, Reader Model (M)=GPT-4.1-nano2025.10 | 20.79 | 32 | |
| RECOMP (Abstractive)Category=Abstractive, Reader Model (M)=GPT-4.1-nano2025.10 | 20.72 | 9.5 | |
| Rerank Top-1Category=Extractive, Reader Model (M)=GPT-4.1-nano2025.10 | 20.2 | 10 | |
| Rerank Top-1Category=Extractive, Reader Model (M)=Llama-3.1-8B-Instruct2025.10 | 16.55 | 10 | |
| BRIEFCategory=Abstractive, Reader Model (M)=Llama-3.1-8B-Instruct2025.10 | 15.8 | 32 |