Multi-hop Question Answering on MuSiQue (EM, F1, Rate)
40EMBRIEF-PRO-LOW
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| BRIEF-PRO-LOWCategory=Abstractive, Reader Model (M)=Llama-3.1-70B-Instruct2025.10 | 40 | 51.95 | 26 | |
| BRIEF-PRO-AUTOCategory=Abstractive, Reader Model (M)=Llama-3.1-70B-Instruct2025.10 | 38.5 | 48.84 | 35 | |
| BRIEF-PRO-HIGHCategory=Abstractive, Reader Model (M)=Llama-3.1-70B-Instruct2025.10 | 38 | 45.55 | 84 | |
| BRIEF-PRO-MEDIUMCategory=Abstractive, Reader Model (M)=Llama-3.1-70B-Instruct2025.10 | 38 | 49.17 | 47 | |
| Non-compressionCategory=Non-compression, Reader Model (M)=Llama-3.1-70B-Instruct2025.10 | 36 | 45.47 | 1 | |
| BRIEF-PRO-AUTO_L7CCategory=Abstractive, Reader Model (M)=Llama-3.1-70B-Instruct2025.10 | 36 | 46.4 | 13 | |
| BRIEF-PRO-LOWCategory=Abstractive, Reader Model (M)=GPT-4.1-nano2025.10 | 33 | 42.96 | 26 | |
| GPT-4.1-nanoCategory=Abstractive, Reader Model (M)=Llama-3.1-70B-Instruct2025.10 | 32 | 41.46 | 128 | |
| LongLLMLinguaCategory=Abstractive, Reader Model (M)=Llama-3.1-70B-Instruct2025.10 | 32 | 42.35 | 10 | |
| GPT-4.1-nanoCategory=Abstractive, Reader Model (M)=GPT-4.1-nano2025.10 | 31.5 | 41.24 | 128 | |
| BRIEF-PRO-HIGHCategory=Abstractive, Reader Model (M)=GPT-4.1-nano2025.10 | 31.5 | 38.7 | 84 | |
| BRIEF-PRO-LOWCategory=Abstractive, Reader Model (M)=Llama-3.1-8B-Instruct2025.10 | 30.5 | 39.68 | 26 | |
| BRIEF-PRO-MEDIUMCategory=Abstractive, Reader Model (M)=GPT-4.1-nano2025.10 | 30.5 | 41.15 | 47 | |
| Rerank Top-5Category=Extractive, Reader Model (M)=Llama-3.1-70B-Instruct2025.10 | 30 | 39.06 | 1.7 | |
| RECOMP (Extractive)Category=Extractive, Reader Model (M)=Llama-3.1-70B-Instruct2025.10 | 29.5 | 37.65 | 41 | |
| BRIEF-PRO-AUTOCategory=Abstractive, Reader Model (M)=GPT-4.1-nano2025.10 | 29.5 | 41.1 | 35 | |
| GPT-4.1-nanoCategory=Abstractive, Reader Model (M)=Llama-3.1-8B-Instruct2025.10 | 28.5 | 40.33 | 128 | |
| BRIEF-PRO-AUTOCategory=Abstractive, Reader Model (M)=Llama-3.1-8B-Instruct2025.10 | 27.5 | 36.64 | 35 | |
| BRIEF-PRO-MEDIUMCategory=Abstractive, Reader Model (M)=Llama-3.1-8B-Instruct2025.10 | 27.5 | 35.55 | 47 | |
| FILM-7BCategory=Long-context LLMs, Reader Model (M)=FILM-7B2025.10 | 26.5 | 36.45 | 1 | |
| Rerank Top-3Category=Extractive, Reader Model (M)=Llama-3.1-70B-Instruct2025.10 | 26 | 35.32 | 2.5 | |
| RECOMP (Abstractive)Category=Abstractive, Reader Model (M)=Llama-3.1-70B-Instruct2025.10 | 25 | 33.8 | 10 | |
| Llama-3.2-3B-InstructCategory=Abstractive, Reader Model (M)=Llama-3.1-70B-Instruct2025.10 | 24.5 | 30.66 | 48 | |
| BRIEF-PRO-HIGHCategory=Abstractive, Reader Model (M)=Llama-3.1-8B-Instruct2025.10 | 24 | 31.7 | 84 | |
| Non-compressionCategory=Non-compression, Reader Model (M)=GPT-4.1-nano2025.10 | 24 | 33.08 | 1 | |
| EXITCategory=Extractive, Reader Model (M)=Llama-3.1-70B-Instruct2025.10 | 23 | 30.85 | 35 | |
| BRIEF-PRO-AUTO_L7CCategory=Abstractive, Reader Model (M)=Llama-3.1-8B-Instruct2025.10 | 22.5 | 32.44 | 13 | |
| Rerank Top-3Category=Extractive, Reader Model (M)=GPT-4.1-nano2025.10 | 22.5 | 30.73 | 2.5 | |
| Rerank Top-5Category=Extractive, Reader Model (M)=GPT-4.1-nano2025.10 | 21 | 29.02 | 1.7 | |
| Non-compressionCategory=Non-compression, Reader Model (M)=Llama-3.1-8B-Instruct2025.10 | 20.5 | 28.88 | 1 | |
| Rerank Top-5Category=Extractive, Reader Model (M)=Llama-3.1-8B-Instruct2025.10 | 20.5 | 28.03 | 1.7 | |
| LongLLMLinguaCategory=Abstractive, Reader Model (M)=Llama-3.1-8B-Instruct2025.10 | 20.5 | 28.75 | 10 | |
| LongLLMLinguaCategory=Abstractive, Reader Model (M)=GPT-4.1-nano2025.10 | 20.5 | 28.55 | 10 | |
| BRIEF-PRO-AUTO_L7CCategory=Abstractive, Reader Model (M)=GPT-4.1-nano2025.10 | 20.5 | 31.58 | 13 | |
| ProLong-8BCategory=Long-context LLMs, Reader Model (M)=ProLong-8B2025.10 | 19.5 | 27.42 | 1 | |
| RECOMP (Extractive)Category=Extractive, Reader Model (M)=GPT-4.1-nano2025.10 | 19.5 | 27.32 | 41 | |
| Rerank Top-1Category=Extractive, Reader Model (M)=Llama-3.1-70B-Instruct2025.10 | 19 | 26.5 | 7 | |
| RECOMP (Abstractive)Category=Abstractive, Reader Model (M)=Llama-3.1-8B-Instruct2025.10 | 17.5 | 25.92 | 10 | |
| Llama-3.2-3B-InstructCategory=Abstractive, Reader Model (M)=GPT-4.1-nano2025.10 | 17.5 | 23.6 | 48 | |
| EXITCategory=Extractive, Reader Model (M)=GPT-4.1-nano2025.10 | 16.5 | 24.33 | 35 | |
| RECOMP (Extractive)Category=Extractive, Reader Model (M)=Llama-3.1-8B-Instruct2025.10 | 16 | 23.39 | 41 | |
| Rerank Top-3Category=Extractive, Reader Model (M)=Llama-3.1-8B-Instruct2025.10 | 15.5 | 23.4 | 2.5 | |
| Llama-3.2-3B-InstructCategory=Abstractive, Reader Model (M)=Llama-3.1-8B-Instruct2025.10 | 13 | 18.48 | 48 | |
| RECOMP (Abstractive)Category=Abstractive, Reader Model (M)=GPT-4.1-nano2025.10 | 13 | 19.13 | 10 | |
| BRIEFCategory=Abstractive, Reader Model (M)=Llama-3.1-70B-Instruct2025.10 | 11.5 | 18.74 | 22 | |
| EXITCategory=Extractive, Reader Model (M)=Llama-3.1-8B-Instruct2025.10 | 11 | 17.19 | 35 | |
| Rerank Top-1Category=Extractive, Reader Model (M)=GPT-4.1-nano2025.10 | 11 | 17.99 | 7 | |
| Rerank Top-1Category=Extractive, Reader Model (M)=Llama-3.1-8B-Instruct2025.10 | 9.5 | 14.58 | 7 | |
| BRIEFCategory=Abstractive, Reader Model (M)=GPT-4.1-nano2025.10 | 6 | 13.55 | 22 | |
| BRIEFCategory=Abstractive, Reader Model (M)=Llama-3.1-8B-Instruct2025.10 | 5 | 12.39 | 22 |