Question Answering on LongBench Qasper
0.4459F1InnerQBase
Evaluation results
| Method | Links | |
|---|---|---|
| InnerQBaseModel=Llama 3.1-8B2026.02 | 0.4459 | |
| BaselineModel=Llama 3.1-8B2026.02 | 0.4458 | |
| KIVIModel=Llama 3.1-8B2026.02 | 0.4408 | |
| KIVISinkModel=Llama 3.1-8B2026.02 | 0.4361 | |
| InnerQHybridModel=Llama 3.1-8B2026.02 | 0.424 | |
| InnerQSmallModel=Llama 3.1-8B2026.02 | 0.4236 | |
| BaselineModel=Llama 3.2-3B2026.02 | 0.4063 | |
| InnerQBaseModel=Llama 3.2-3B2026.02 | 0.4032 | |
| InnerQHybridModel=Llama 3.2-3B2026.02 | 0.3959 | |
| InnerQSmallModel=Llama 3.2-3B2026.02 | 0.3884 | |
| KIVISinkModel=Llama 3.2-3B2026.02 | 0.3849 | |
| KIVIModel=Llama 3.2-3B2026.02 | 0.3709 | |
| PICToken Budget=128×52026.01 | 0.2838 | |
| M-RAG†-PToken Budget=128×5, Prompting=Few-shot, Sorting Strategy=Position sorting2026.01 | 0.2693 | |
| M-RAG†-SToken Budget=128×5, Prompting=Few-shot, Sorting Strategy=Similarity sorting2026.01 | 0.269 | |
| SemanticToken Budget=128×52026.01 | 0.2683 | |
| M-RAG-SToken Budget=128×5, Prompting=Zero-shot, Sorting Strategy=Similarity sorting2026.01 | 0.2655 | |
| DOSToken Budget=128×52026.01 | 0.2635 | |
| PICToken Budget=128×32026.01 | 0.253 | |
| SemanticToken Budget=128×32026.01 | 0.2482 | |
| M-RAG-PToken Budget=128×5, Prompting=Zero-shot, Sorting Strategy=Position sorting2026.01 | 0.2456 | |
| Fixed-SizeToken Budget=128×52026.01 | 0.2425 | |
| M-RAG†-PToken Budget=128×3, Prompting=Few-shot, Sorting Strategy=Position sorting2026.01 | 0.237 | |
| M-RAG-PToken Budget=128×3, Prompting=Zero-shot, Sorting Strategy=Position sorting2026.01 | 0.2317 | |
| M-RAG-SToken Budget=128×3, Prompting=Zero-shot, Sorting Strategy=Similarity sorting2026.01 | 0.2308 | |
| DOSToken Budget=128×32026.01 | 0.2269 | |
| M-RAG†-SToken Budget=128×3, Prompting=Few-shot, Sorting Strategy=Similarity sorting2026.01 | 0.2224 | |
| InnerQHybridModel=Llama 2-7B2026.02 | 0.2218 | |
| InnerQBaseModel=Llama 2-7B2026.02 | 0.2184 | |
| KIVIModel=Llama 2-7B2026.02 | 0.2111 | |
| BaselineModel=Llama 2-7B2026.02 | 0.2099 | |
| InnerQHybridModel=Llama 3.2-1B2026.02 | 0.2043 | |
| KIVISinkModel=Llama 2-7B2026.02 | 0.203 | |
| BaselineModel=Llama 3.2-1B2026.02 | 0.2029 | |
| InnerQSmallModel=Llama 3.2-1B2026.02 | 0.1984 | |
| Fixed-SizeToken Budget=128×32026.01 | 0.1908 | |
| InnerQBaseModel=Llama 3.2-1B2026.02 | 0.1837 | |
| M-RAG†-SToken Budget=128×1, Prompting=Few-shot, Sorting Strategy=Similarity sorting2026.01 | 0.182 | |
| M-RAG†-PToken Budget=128×1, Prompting=Few-shot, Sorting Strategy=Position sorting2026.01 | 0.1806 | |
| KIVIModel=Llama 3.2-1B2026.02 | 0.18 | |
| InnerQBaseModel=Llama 2-13B2026.02 | 0.1785 | |
| InnerQSmallModel=Llama 2-13B2026.02 | 0.175 | |
| KIVISinkModel=Llama 3.2-1B2026.02 | 0.1723 | |
| BaselineModel=Llama 2-13B2026.02 | 0.1659 | |
| InnerQHybridModel=Llama 2-13B2026.02 | 0.1656 | |
| SemanticToken Budget=128×12026.01 | 0.1654 | |
| DOSToken Budget=128×12026.01 | 0.1643 | |
| PICToken Budget=128×12026.01 | 0.1581 | |
| KIVISinkModel=Llama 2-13B2026.02 | 0.1548 | |
| KIVIModel=Llama 2-13B2026.02 | 0.1509 | |
| M-RAG-PToken Budget=128×1, Prompting=Zero-shot, Sorting Strategy=Position sorting2026.01 | 0.1436 | |
| M-RAG-SToken Budget=128×1, Prompting=Zero-shot, Sorting Strategy=Similarity sorting2026.01 | 0.139 | |
| Fixed-SizeToken Budget=128×12026.01 | 0.1316 | |
| InnerQSmallModel=Llama 2-7B2026.02 | 0.1312 | |
| CurvPruneContext Budget (B)=20482026.02 | 0.102 | |
| BM25Context Budget (B)=20482026.02 | 0.095 | |
| BM25+TexContext Budget (B)=20482026.02 | 0.093 | |
| Head+TailContext Budget (B)=20482026.02 | 0.082 | |
| RecencyContext Budget (B)=20482026.02 | 0.077 | |
| Sel. CtxContext Budget (B)=20482026.02 | 0.074 | |
| RandomContext Budget (B)=20482026.02 | 0.063 | |
| LLMLinguaContext Budget (B)=20482026.02 | 0.059 |