Multi-hop Question Answering on LongSeal
15.35EMNon-compression
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| Non-compressionCategory=Non-compression, Reader Model (M)=Llama-3.1-70B-Instruct2025.10 | 15.35 | 20.05 | 1 | |
| BRIEF-PRO-AUTOCategory=Abstractive, Reader Model (M)=Llama-3.1-8B-Instruct2025.10 | 12.99 | 17.62 | 36 | |
| BRIEF-PRO-AUTOCategory=Abstractive, Reader Model (M)=Llama-3.1-70B-Instruct2025.10 | 12.99 | 17.59 | 36 | |
| Non-compressionCategory=Non-compression, Reader Model (M)=Llama-3.1-8B-Instruct2025.10 | 12.6 | 16.81 | 1 | |
| BRIEF-PRO-LOWCategory=Abstractive, Reader Model (M)=Llama-3.1-70B-Instruct2025.10 | 12.6 | 16.72 | 31 | |
| BRIEF-PRO-AUTOCategory=Abstractive, Reader Model (M)=GPT-4.1-nano2025.10 | 12.2 | 16.87 | 36 | |
| BRIEF-PRO-LOWCategory=Abstractive, Reader Model (M)=Llama-3.1-8B-Instruct2025.10 | 11.42 | 16.63 | 31 | |
| BRIEF-PRO-LOWCategory=Abstractive, Reader Model (M)=GPT-4.1-nano2025.10 | 11.42 | 15.76 | 31 | |
| Rerank Top-5Category=Extractive, Reader Model (M)=Llama-3.1-70B-Instruct2025.10 | 11.02 | 14.57 | 2.7 | |
| Non-compressionCategory=Non-compression, Reader Model (M)=GPT-4.1-nano2025.10 | 11.02 | 16.02 | 1 | |
| GPT-4.1-nanoCategory=Abstractive, Reader Model (M)=GPT-4.1-nano2025.10 | 11.02 | 16.51 | 116 | |
| BRIEF-PRO-HIGHCategory=Abstractive, Reader Model (M)=GPT-4.1-nano2025.10 | 11.02 | 14.91 | 72 | |
| BRIEF-PRO-MEDIUMCategory=Abstractive, Reader Model (M)=GPT-4.1-nano2025.10 | 11.02 | 15.31 | 52 | |
| GPT-4.1-nanoCategory=Abstractive, Reader Model (M)=Llama-3.1-8B-Instruct2025.10 | 10.63 | 15.95 | 116 | |
| Rerank Top-3Category=Extractive, Reader Model (M)=Llama-3.1-70B-Instruct2025.10 | 10.63 | 13.21 | 4.2 | |
| GPT-4.1-nanoCategory=Abstractive, Reader Model (M)=Llama-3.1-70B-Instruct2025.10 | 10.63 | 16.55 | 116 | |
| BRIEF-PRO-MEDIUMCategory=Abstractive, Reader Model (M)=Llama-3.1-70B-Instruct2025.10 | 10.63 | 16.41 | 52 | |
| BRIEF-PRO-MEDIUMCategory=Abstractive, Reader Model (M)=Llama-3.1-8B-Instruct2025.10 | 10.24 | 15.67 | 52 | |
| LongLLMLinguaCategory=Abstractive, Reader Model (M)=Llama-3.1-70B-Instruct2025.10 | 10.24 | 14.87 | 14 | |
| FILM-7BCategory=Long-context LLMs, Reader Model (M)=FILM-7B2025.10 | 9.84 | 15.1 | 1 | |
| Llama-3.2-3B-InstructCategory=Abstractive, Reader Model (M)=Llama-3.1-70B-Instruct2025.10 | 9.45 | 12.48 | 58 | |
| BRIEF-PRO-HIGHCategory=Abstractive, Reader Model (M)=Llama-3.1-70B-Instruct2025.10 | 9.45 | 14.18 | 72 | |
| Rerank Top-5Category=Extractive, Reader Model (M)=GPT-4.1-nano2025.10 | 9.45 | 13.99 | 2.7 | |
| Rerank Top-5Category=Extractive, Reader Model (M)=Llama-3.1-8B-Instruct2025.10 | 9.06 | 12.42 | 2.7 | |
| BRIEF-PRO-HIGHCategory=Abstractive, Reader Model (M)=Llama-3.1-8B-Instruct2025.10 | 9.06 | 13.73 | 72 | |
| BRIEF-PRO-AUTO_L7CCategory=Abstractive, Reader Model (M)=Llama-3.1-70B-Instruct2025.10 | 9.06 | 12.68 | 11 | |
| LongLLMLinguaCategory=Abstractive, Reader Model (M)=Llama-3.1-8B-Instruct2025.10 | 8.27 | 13.26 | 14 | |
| BRIEFCategory=Abstractive, Reader Model (M)=GPT-4.1-nano2025.10 | 8.27 | 12.3 | 19 | |
| Llama-3.2-3B-InstructCategory=Abstractive, Reader Model (M)=GPT-4.1-nano2025.10 | 8.27 | 11.76 | 58 | |
| Rerank Top-3Category=Extractive, Reader Model (M)=Llama-3.1-8B-Instruct2025.10 | 7.87 | 12.19 | 4.2 | |
| EXITCategory=Extractive, Reader Model (M)=Llama-3.1-70B-Instruct2025.10 | 7.87 | 11.19 | 20 | |
| Rerank Top-3Category=Extractive, Reader Model (M)=GPT-4.1-nano2025.10 | 7.87 | 11.83 | 4.2 | |
| BRIEF-PRO-AUTO_L7CCategory=Abstractive, Reader Model (M)=GPT-4.1-nano2025.10 | 7.87 | 11.35 | 11 | |
| Llama-3.2-3B-InstructCategory=Abstractive, Reader Model (M)=Llama-3.1-8B-Instruct2025.10 | 7.48 | 12.03 | 58 | |
| ProLong-8BCategory=Long-context LLMs, Reader Model (M)=ProLong-8B2025.10 | 7.09 | 11.71 | 1 | |
| EXITCategory=Extractive, Reader Model (M)=Llama-3.1-8B-Instruct2025.10 | 7.09 | 10.22 | 20 | |
| BRIEFCategory=Abstractive, Reader Model (M)=Llama-3.1-70B-Instruct2025.10 | 7.09 | 11.13 | 19 | |
| RECOMP (Abstractive)Category=Abstractive, Reader Model (M)=Llama-3.1-70B-Instruct2025.10 | 7.09 | 11.1 | 9 | |
| Rerank Top-1Category=Extractive, Reader Model (M)=Llama-3.1-70B-Instruct2025.10 | 6.69 | 9.23 | 15 | |
| EXITCategory=Extractive, Reader Model (M)=GPT-4.1-nano2025.10 | 6.69 | 10.65 | 20 | |
| Rerank Top-1Category=Extractive, Reader Model (M)=Llama-3.1-8B-Instruct2025.10 | 6.3 | 9.08 | 15 | |
| BRIEF-PRO-AUTO_L7CCategory=Abstractive, Reader Model (M)=Llama-3.1-8B-Instruct2025.10 | 6.12 | 10.59 | 11 | |
| BRIEFCategory=Abstractive, Reader Model (M)=Llama-3.1-8B-Instruct2025.10 | 5.91 | 10.02 | 19 | |
| RECOMP (Extractive)Category=Extractive, Reader Model (M)=Llama-3.1-70B-Instruct2025.10 | 5.91 | 9.43 | 35 | |
| LongLLMLinguaCategory=Abstractive, Reader Model (M)=GPT-4.1-nano2025.10 | 5.91 | 11.35 | 14 | |
| RECOMP (Abstractive)Category=Abstractive, Reader Model (M)=Llama-3.1-8B-Instruct2025.10 | 5.51 | 9.88 | 9 | |
| RECOMP (Extractive)Category=Extractive, Reader Model (M)=Llama-3.1-8B-Instruct2025.10 | 4.72 | 8.18 | 35 | |
| Rerank Top-1Category=Extractive, Reader Model (M)=GPT-4.1-nano2025.10 | 3.94 | 6.55 | 15 | |
| RECOMP (Extractive)Category=Extractive, Reader Model (M)=GPT-4.1-nano2025.10 | 3.54 | 7.06 | 35 | |
| RECOMP (Abstractive)Category=Abstractive, Reader Model (M)=GPT-4.1-nano2025.10 | 3.54 | 8.11 | 9 |