Generative sense-making QA on LongBench
0.6573ComprehensivenessHGMEM
Evaluation results
| Method | Links | ||
|---|---|---|---|
| HGMEMBase LLM=GPT-4o, Type=Ours, Working Memory=true2025.12 | 0.6573 | 0.6974 | |
| HGMEMBase LLM=Qwen2.5-32B-Instruct, Type=Ours, Working Memory=true2025.12 | 0.6418 | 0.6651 | |
| DeepRAGBase LLM=GPT-4o, Type=Multi-step RAG, Working Memory=true2025.12 | 0.6362 | 0.6598 | |
| ComoRAGBase LLM=GPT-4o, Type=Multi-step RAG, Working Memory=true2025.12 | 0.6218 | 0.6582 | |
| NaiveRAGBase LLM=GPT-4o, Type=Traditional RAG, Working Memory=false2025.12 | 0.6162 | 0.642 | |
| LightRAGBase LLM=GPT-4o, Type=Traditional RAG, Working Memory=false2025.12 | 0.6155 | 0.6337 | |
| DeepRAGBase LLM=Qwen2.5-32B-Instruct, Type=Multi-step RAG, Working Memory=true2025.12 | 0.6145 | 0.6356 | |
| NaiveRAGBase LLM=Qwen2.5-32B-Instruct, Type=Traditional RAG, Working Memory=false2025.12 | 0.6141 | 0.6225 | |
| LightRAGBase LLM=Qwen2.5-32B-Instruct, Type=Traditional RAG, Working Memory=false2025.12 | 0.6082 | 0.6273 | |
| GraphRAGBase LLM=Qwen2.5-32B-Instruct, Type=Traditional RAG, Working Memory=false2025.12 | 0.6078 | 0.6216 | |
| ComoRAGBase LLM=Qwen2.5-32B-Instruct, Type=Multi-step RAG, Working Memory=true2025.12 | 0.6074 | 0.6128 | |
| GraphRAGBase LLM=GPT-4o, Type=Traditional RAG, Working Memory=false2025.12 | 0.6039 | 0.6402 | |
| HippoRAG v2Base LLM=GPT-4o, Type=Traditional RAG, Working Memory=false2025.12 | 0.5892 | 0.6127 | |
| HippoRAG v2Base LLM=Qwen2.5-32B-Instruct, Type=Traditional RAG, Working Memory=false2025.12 | 0.5666 | 0.608 |