Summarization on LongBench GovReport (ROUGE-L)
34.61ROUGE-LBaseline
Evaluation Results
| Method | Links | |
|---|---|---|
| BaselineModel=Llama 3.1-8B2026.02 | 34.61 | |
| KIVIModel=Llama 3.1-8B2026.02 | 34.47 | |
| KIVISinkModel=Llama 3.1-8B2026.02 | 34.33 | |
| InnerQBaseModel=Llama 3.1-8B2026.02 | 34.33 | |
| BaselineModel=Llama 3.2-3B2026.02 | 33.86 | |
| InnerQSmallModel=Llama 3.1-8B2026.02 | 33.8 | |
| KIVISinkModel=Llama 3.2-3B2026.02 | 33.76 | |
| InnerQHybridModel=Llama 3.1-8B2026.02 | 33.62 | |
| InnerQBaseModel=Llama 3.2-3B2026.02 | 33.38 | |
| InnerQHybridModel=Llama 3.2-3B2026.02 | 32.73 | |
| InnerQSmallModel=Llama 3.2-3B2026.02 | 32.25 | |
| KIVIModel=Llama 3.2-3B2026.02 | 32.11 | |
| BaselineModel=Llama 3.2-1B2026.02 | 29.27 | |
| InnerQBaseModel=Llama 3.2-1B2026.02 | 28.89 | |
| KIVISinkModel=Llama 3.2-1B2026.02 | 28.87 | |
| KIVIModel=Llama 3.2-1B2026.02 | 28.57 | |
| BaselineModel=Llama 2-13B2026.02 | 27.82 | |
| InnerQBaseModel=Llama 2-13B2026.02 | 27.29 | |
| KIVISinkModel=Llama 2-13B2026.02 | 27.1 | |
| KIVIModel=Llama 2-13B2026.02 | 27.01 | |
| BaselineModel=Llama 2-7B2026.02 | 26.88 | |
| InnerQHybridModel=Llama 3.2-1B2026.02 | 26.72 | |
| InnerQSmallModel=Llama 3.2-1B2026.02 | 26.7 | |
| KIVIModel=Llama 2-7B2026.02 | 26.68 | |
| KIVISinkModel=Llama 2-7B2026.02 | 26.64 | |
| InnerQHybridModel=Llama 2-13B2026.02 | 26.38 | |
| InnerQBaseModel=Llama 2-7B2026.02 | 26.12 | |
| InnerQSmallModel=Llama 2-13B2026.02 | 25.69 | |
| InnerQHybridModel=Llama 2-7B2026.02 | 25.18 | |
| CurvPruneContext Budget (B)=20482026.02 | 21 | |
| RandomContext Budget (B)=20482026.02 | 20.6 | |
| BM25+TexContext Budget (B)=20482026.02 | 20.5 | |
| Head+TailContext Budget (B)=20482026.02 | 20.4 | |
| BM25Context Budget (B)=20482026.02 | 20.3 | |
| LLMLinguaContext Budget (B)=20482026.02 | 19.8 | |
| Sel. CtxContext Budget (B)=20482026.02 | 19.7 | |
| RecencyContext Budget (B)=20482026.02 | 19.1 | |
| InnerQSmallModel=Llama 2-7B2026.02 | 17.15 |