Long-document Understanding Benchmark
83Synonymy ScoreHMARS
Evaluation results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| HMARSP (Page size)=10242026.06 | 83 | 66.9 | 70.8 | 91.7 | 92 | |
| RAGK (Retrieval budget)=20, Retrieval chunk size=1K2026.06 | 82.2 | 66 | 60.7 | 82.1 | 94.7 | |
| Dense RerankN (Rerank candidates)=1002026.06 | 78.1 | 66 | 64.3 | 78.6 | 89.5 | |
| Dense RerankN (Rerank candidates)=502026.06 | 76.7 | 63.8 | 67.9 | 71.4 | 84.2 | |
| RAGK (Retrieval budget)=50, Retrieval chunk size=1K2026.06 | 75.4 | 57.5 | 68 | 75.1 | 79 | |
| Full-Context LLM2026.06 | 74 | 59.6 | 53.6 | 75 | 84.2 | |
| RAGK (Retrieval budget)=10, Retrieval chunk size=1K2026.06 | 65.8 | 53.2 | 60.7 | 71.4 | 68.4 | |
| GraphRAG2026.06 | 64.4 | 57.4 | 64.3 | 67.9 | 57.9 | |
| RAGK (Retrieval budget)=3, Retrieval chunk size=4K2026.06 | 56.2 | 29.8 | 42.9 | 53.6 | 47.4 | |
| RAGK (Retrieval budget)=5, Retrieval chunk size=4K2026.06 | 56.1 | 29.8 | 53.5 | 57 | 57.8 | |
| CoA-7BModel parameters=7B2026.06 | 5.5 | 8.5 | 7.1 | 10.8 | 0 | |
| MemAgent-14BModel parameters=14B2026.06 | 2.6 | 2.1 | 7.2 | 3.6 | 0 |