Long narrative understanding QA on Prelude
62.96AccuracyHGMEM
Evaluation results
| Method | Links | |
|---|---|---|
| HGMEMBase LLM=GPT-4o, Type=Ours, Working Memory=true2025.12 | 62.96 | |
| HGMEMBase LLM=Qwen2.5-32B-Instruct, Type=Ours, Working Memory=true2025.12 | 62.22 | |
| LightRAGBase LLM=GPT-4o, Type=Traditional RAG, Working Memory=false2025.12 | 61.48 | |
| LightRAGBase LLM=Qwen2.5-32B-Instruct, Type=Traditional RAG, Working Memory=false2025.12 | 60.74 | |
| NaiveRAGBase LLM=GPT-4o, Type=Traditional RAG, Working Memory=false2025.12 | 60 | |
| GraphRAGBase LLM=GPT-4o, Type=Traditional RAG, Working Memory=false2025.12 | 59.26 | |
| DeepRAGBase LLM=GPT-4o, Type=Multi-step RAG, Working Memory=true2025.12 | 56.3 | |
| HippoRAG v2Base LLM=GPT-4o, Type=Traditional RAG, Working Memory=false2025.12 | 54.81 | |
| ComoRAGBase LLM=GPT-4o, Type=Multi-step RAG, Working Memory=true2025.12 | 54.07 | |
| NaiveRAGBase LLM=Qwen2.5-32B-Instruct, Type=Traditional RAG, Working Memory=false2025.12 | 52.59 | |
| HippoRAG v2Base LLM=Qwen2.5-32B-Instruct, Type=Traditional RAG, Working Memory=false2025.12 | 51.85 | |
| DeepRAGBase LLM=Qwen2.5-32B-Instruct, Type=Multi-step RAG, Working Memory=true2025.12 | 51.11 | |
| GraphRAGBase LLM=Qwen2.5-32B-Instruct, Type=Traditional RAG, Working Memory=false2025.12 | 50.37 | |
| ComoRAGBase LLM=Qwen2.5-32B-Instruct, Type=Multi-step RAG, Working Memory=true2025.12 | 50.37 |