Summarization on MultiNews LongBench (test)
30.91ROUGE-L ScoreMemAgent-7B
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| MemAgent-7Bsetting=zero-shot2026.02 | 30.91 | 48.49 | 14.41 | 31.27 | |
| LycheeMemory-7Bsetting=zero-shot2026.02 | 30.33 | 47.43 | 15.28 | 31.01 | |
| Qwen2.5-7B-Instructsetting=zero-shot2026.02 | 28.33 | 46.64 | 12.01 | 28.99 | |
| BaselineModel=Llama 3.1-8B2026.02 | 26.95 | — | — | — | |
| KIVISinkModel=Llama 3.1-8B2026.02 | 26.76 | — | — | — | |
| InnerQBaseModel=Llama 3.1-8B2026.02 | 26.73 | — | — | — | |
| BaselineModel=Llama 2-13B2026.02 | 26.56 | — | — | — | |
| KIVIModel=Llama 3.1-8B2026.02 | 26.39 | — | — | — | |
| KIVISinkModel=Llama 2-13B2026.02 | 26.26 | — | — | — | |
| InnerQHybridModel=Llama 3.1-8B2026.02 | 26.2 | — | — | — | |
| InnerQHybridModel=Llama 2-7B2026.02 | 26.18 | — | — | — | |
| BaselineModel=Llama 2-7B2026.02 | 26.17 | — | — | — | |
| InnerQBaseModel=Llama 2-13B2026.02 | 26.17 | — | — | — | |
| InnerQSmallModel=Llama 2-13B2026.02 | 26.16 | — | — | — | |
| InnerQSmallModel=Llama 3.1-8B2026.02 | 26.09 | — | — | — | |
| BaselineModel=Llama 3.2-1B2026.02 | 26.08 | — | — | — | |
| InnerQBaseModel=Llama 3.2-1B2026.02 | 26.07 | — | — | — | |
| KIVISinkModel=Llama 3.2-3B2026.02 | 26.07 | — | — | — | |
| KIVISinkModel=Llama 3.2-1B2026.02 | 26.02 | — | — | — | |
| InnerQHybridModel=Llama 2-13B2026.02 | 25.97 | — | — | — | |
| BaselineModel=Llama 3.2-3B2026.02 | 25.93 | — | — | — | |
| KIVIModel=Llama 2-7B2026.02 | 25.93 | — | — | — | |
| KIVIModel=Llama 3.2-1B2026.02 | 25.89 | — | — | — | |
| KIVISinkModel=Llama 2-7B2026.02 | 25.8 | — | — | — | |
| InnerQBaseModel=Llama 2-7B2026.02 | 25.8 | — | — | — | |
| KIVIModel=Llama 2-13B2026.02 | 25.78 | — | — | — | |
| KIVIModel=Llama 3.2-3B2026.02 | 25.74 | — | — | — | |
| InnerQBaseModel=Llama 3.2-3B2026.02 | 25.7 | — | — | — | |
| InnerQHybridModel=Llama 3.2-3B2026.02 | 25.43 | — | — | — | |
| InnerQSmallModel=Llama 3.2-3B2026.02 | 25.35 | — | — | — | |
| InnerQHybridModel=Llama 3.2-1B2026.02 | 25.06 | — | — | — | |
| InnerQSmallModel=Llama 3.2-1B2026.02 | 24.7 | — | — | — | |
| InnerQSmallModel=Llama 2-7B2026.02 | 11.7 | — | — | — |