Question Answering on MultiFieldQA en
24.67EMBRIEF-PRO-AUTO
Evaluation Results
| Method | Links | |||
|---|---|---|---|---|
| BRIEF-PRO-AUTOBackbone LM=Llama-3.1-8B-Instruct, Category=Abstractive2025.10 | 24.67 | 54.21 | 11 | |
| BRIEF-PRO-HIGHBackbone LM=Llama-3.1-8B-Instruct, Category=Abstractive2025.10 | 24 | 52.43 | 29 | |
| BRIEF-PRO-MEDIUMBackbone LM=Llama-3.1-8B-Instruct, Category=Abstractive2025.10 | 24 | 52.23 | 16 | |
| BRIEF-PRO-LOWBackbone LM=Llama-3.1-8B-Instruct, Category=Abstractive2025.10 | 24 | 53.88 | 10 | |
| ProLong-8BCategory=Long-context LLMs2025.10 | 22 | 51.44 | 1 | |
| FILM-7BCategory=Long-context LLMs2025.10 | 22 | 54.34 | 1 | |
| Non-compressionBackbone LM=Llama-3.1-8B-Instruct, Category=Non-compression2025.10 | 21.33 | 52.77 | 1 | |
| Non-compressionBackbone LM=Llama-3.1-70B-Instruct, Category=Non-compression2025.10 | 21.33 | 54.38 | 1 | |
| BRIEF-PRO-HIGHBackbone LM=Llama-3.1-70B-Instruct, Category=Abstractive2025.10 | 21.33 | 50.7 | 29 | |
| BRIEF-PRO-AUTOBackbone LM=Llama-3.1-70B-Instruct, Category=Abstractive2025.10 | 20.67 | 51.16 | 11 | |
| BRIEF-PRO-MEDIUMBackbone LM=Llama-3.1-70B-Instruct, Category=Abstractive2025.10 | 20.67 | 51.93 | 16 | |
| BRIEF-PRO-LOWBackbone LM=Llama-3.1-70B-Instruct, Category=Abstractive2025.10 | 20.67 | 50.63 | 10 | |
| RECOMP (Extractive)Backbone LM=Llama-3.1-70B-Instruct, Category=Extractive2025.10 | 18 | 46.43 | 16 | |
| BRIEF-PRO-HIGHBackbone LM=GPT-4.1-nano, Category=Abstractive2025.10 | 18 | 48.53 | 29 | |
| RECOMP (Extractive)Backbone LM=Llama-3.1-8B-Instruct, Category=Extractive2025.10 | 17.33 | 43.82 | 16 | |
| LongLLMLinguaBackbone LM=Llama-3.1-8B-Instruct, Category=Abstractive2025.10 | 17.33 | 44.62 | 4 | |
| Rerank Top-3Backbone LM=Llama-3.1-70B-Instruct, Category=Extractive2025.10 | 17.33 | 45.1 | 3 | |
| BRIEF-PRO-LOWBackbone LM=GPT-4.1-nano, Category=Abstractive2025.10 | 17.33 | 49.75 | 10 | |
| LongLLMLinguaBackbone LM=Llama-3.1-70B-Instruct, Category=Abstractive2025.10 | 16.67 | 45.87 | 4 | |
| Non-compressionBackbone LM=GPT-4.1-nano, Category=Non-compression2025.10 | 16.67 | 50.09 | 1 | |
| BRIEF-PRO-AUTOBackbone LM=GPT-4.1-nano, Category=Abstractive2025.10 | 16.67 | 48.17 | 11 | |
| BRIEF-PRO-MEDIUMBackbone LM=GPT-4.1-nano, Category=Abstractive2025.10 | 16.67 | 49.79 | 16 | |
| Rerank Top-3Backbone LM=Llama-3.1-8B-Instruct, Category=Extractive2025.10 | 16.5 | 43.1 | 3 | |
| Llama-3.2-3B-InstructBackbone LM=Llama-3.1-8B-Instruct, Category=Abstractive2025.10 | 14.67 | 42.3 | 16 | |
| GPT-4.1-nanoBackbone LM=Llama-3.1-8B-Instruct, Category=Abstractive2025.10 | 14.67 | 41.95 | 29 | |
| LongLLMLinguaBackbone LM=GPT-4.1-nano, Category=Abstractive2025.10 | 14 | 44.88 | 4 | |
| GPT-4.1-nanoBackbone LM=Llama-3.1-70B-Instruct, Category=Abstractive2025.10 | 13.33 | 41.44 | 29 | |
| Rerank Top-3Backbone LM=GPT-4.1-nano, Category=Extractive2025.10 | 13.33 | 42.67 | 3 | |
| Llama-3.2-3B-InstructBackbone LM=Llama-3.1-70B-Instruct, Category=Abstractive2025.10 | 12.8 | 39.31 | 16 | |
| RECOMP (Extractive)Backbone LM=GPT-4.1-nano, Category=Extractive2025.10 | 12.67 | 43.72 | 16 | |
| Llama-3.2-3B-InstructBackbone LM=GPT-4.1-nano, Category=Abstractive2025.10 | 12.67 | 41.55 | 16 | |
| GPT-4.1-nanoBackbone LM=GPT-4.1-nano, Category=Abstractive2025.10 | 12.67 | 41.04 | 29 | |
| RECOMP (Abstractive)Backbone LM=Llama-3.1-70B-Instruct, Category=Abstractive2025.10 | 11.33 | 33.78 | 12 | |
| EXITBackbone LM=Llama-3.1-8B-Instruct, Category=Extractive2025.10 | 10.67 | 25.33 | 14 | |
| EXITBackbone LM=Llama-3.1-70B-Instruct, Category=Extractive2025.10 | 10.67 | 25.75 | 14 | |
| EXITBackbone LM=GPT-4.1-nano, Category=Extractive2025.10 | 9.33 | 27.12 | 14 | |
| RECOMP (Abstractive)Backbone LM=Llama-3.1-8B-Instruct, Category=Abstractive2025.10 | 8 | 30.14 | 12 | |
| RECOMP (Abstractive)Backbone LM=GPT-4.1-nano, Category=Abstractive2025.10 | 7.33 | 28.83 | 12 |