Long narrative understanding QA on NoCha
65.1Pair AccuracyMiA-RAG
Evaluation results
| Method | Links | ||
|---|---|---|---|
| MiA-RAGRetriever=MiA-Emb (+sig), Generator=DS-V3.2 (+sig)2026.05 | 65.1 | 82.5 | |
| Query-only RAGRetriever=MiA-Emb, Generator=DS-V3.22026.05 | 61.9 | 81 | |
| MiA-EmbRetriever=MiA-Emb (+sig), Generator=DS-V3.22026.05 | 58.7 | 79.4 | |
| MiA-RAGRetriever=MiA-Emb-8B, +S (Retriever)=true, Generator=MiA-Gen-14B, +S (Generator)=true, top-k=32025.12 | 55.56 | — | |
| MiA-RAGRetriever=MiA-Emb-8B, +S (Retriever)=true, Generator=MiA-Gen-14B, +S (Generator)=true, top-k=102025.12 | 53.97 | — | |
| MiARetriever=MiA-Emb-8B, +S (Retriever)=true, Generator=Qwen2.5-72B, +S (Generator)=true, top-k=102025.12 | 52.38 | — | |
| MiA (Gen-Only)Retriever=Qwen3-8B, +S (Retriever)=false, Generator=MiA-Gen-14B, +S (Generator)=true, top-k=102025.12 | 50.79 | — | |
| MiA (Emb-Only)Retriever=MiA-Emb-8B, +S (Retriever)=true, Generator=Qwen2.5-72B, +S (Generator)=false, top-k=102025.12 | 49.21 | — | |
| MiA (Gen-Only)Retriever=Qwen3-8B, +S (Retriever)=false, Generator=MiA-Gen-14B, +S (Generator)=true, top-k=32025.12 | 49.21 | — | |
| MiA-RAGRetriever=MiA-Emb-8B, +S (Retriever)=true, Generator=MiA-Gen-14B, +S (Generator)=true, top-k=52025.12 | 49.21 | — | |
| Query-only RAGRetriever=Qwen3-Emb, Generator=DS-V3.22026.05 | 49.2 | 74.6 | |
| MiA (Gen-Only)Retriever=Qwen3-8B, +S (Retriever)=false, Generator=MiA-Gen-14B, +S (Generator)=true, top-k=52025.12 | 47.62 | — | |
| MiA-RAGRetriever=MiA-Emb (+sig), Generator=MiA-Gen-14B (+sig)2026.05 | 47.6 | 71.4 | |
| MiA (Gen-Only)Retriever=Qwen3-8B, +S (Retriever)=false, Generator=Qwen2.5-72B, +S (Generator)=true, top-k=102025.12 | 44.44 | — | |
| MiARetriever=MiA-Emb-8B, +S (Retriever)=true, Generator=Qwen2.5-72B, +S (Generator)=true, top-k=52025.12 | 44.44 | — | |
| MiA (Gen-Only)Retriever=Qwen3-8B, +S (Retriever)=false, Generator=Qwen2.5-72B, +S (Generator)=true, top-k=52025.12 | 42.86 | — | |
| MiA (Emb-Only)Retriever=MiA-Emb-8B, +S (Retriever)=true, Generator=Qwen2.5-72B, +S (Generator)=false, top-k=32025.12 | 42.86 | — | |
| MiA (Emb-Only)Retriever=MiA-Emb-8B, +S (Retriever)=true, Generator=Qwen2.5-72B, +S (Generator)=false, top-k=52025.12 | 42.86 | — | |
| VanillaRetriever=Qwen3-8B, +S (Retriever)=false, Generator=Qwen2.5-72B, +S (Generator)=false, top-k=102025.12 | 41.27 | — | |
| MiARetriever=MiA-Emb-8B, +S (Retriever)=true, Generator=Qwen2.5-72B, +S (Generator)=true, top-k=32025.12 | 41.27 | — | |
| VanillaRetriever=Qwen3-8B, +S (Retriever)=false, Generator=Qwen2.5-72B, +S (Generator)=false, top-k=52025.12 | 38.1 | — | |
| MiARetriever=MiA-Emb-8B, +S (Retriever)=true, Generator=Qwen2.5-14B, +S (Generator)=true, top-k=102025.12 | 38.1 | — | |
| MiA (Gen-Only)Retriever=Qwen3-8B, +S (Retriever)=false, Generator=Qwen2.5-72B, +S (Generator)=true, top-k=32025.12 | 36.51 | — | |
| VanillaRetriever=Qwen3-8B, +S (Retriever)=false, Generator=Qwen2.5-72B, +S (Generator)=false, top-k=32025.12 | 33.33 | — | |
| Query-only RAGRetriever=Qwen3-Emb, Generator=Qwen-14B2026.05 | 31.8 | 64.3 | |
| Summary-OnlyGenerator=Qwen2.5-72B, +S (Generator)=true2025.12 | 31.75 | — | |
| MiARetriever=MiA-Emb-8B, +S (Retriever)=true, Generator=Qwen2.5-14B, +S (Generator)=true, top-k=32025.12 | 30.16 | — | |
| MiARetriever=MiA-Emb-8B, +S (Retriever)=true, Generator=Qwen2.5-14B, +S (Generator)=true, top-k=52025.12 | 28.57 | — | |
| MiA (Gen-Only)Retriever=Qwen3-8B, +S (Retriever)=false, Generator=Qwen2.5-14B, +S (Generator)=true, top-k=102025.12 | 26.98 | — | |
| MiA (Emb-Only)Retriever=MiA-Emb-8B, +S (Retriever)=true, Generator=Qwen2.5-14B, +S (Generator)=false, top-k=52025.12 | 26.98 | — | |
| MiA (Emb-Only)Retriever=MiA-Emb-8B, +S (Retriever)=true, Generator=Qwen2.5-14B, +S (Generator)=false, top-k=102025.12 | 26.98 | — | |
| MiA (Emb-Only)Retriever=MiA-Emb-8B, +S (Retriever)=true, Generator=Qwen2.5-14B, +S (Generator)=false, top-k=32025.12 | 22.22 | — | |
| MiA (Gen-Only)Retriever=Qwen3-8B, +S (Retriever)=false, Generator=Qwen2.5-14B, +S (Generator)=true, top-k=32025.12 | 19.05 | — | |
| Summary-OnlyGenerator=Qwen2.5-14B, +S (Generator)=true2025.12 | 17.46 | — | |
| VanillaRetriever=Qwen3-8B, +S (Retriever)=false, Generator=Qwen2.5-14B, +S (Generator)=false, top-k=32025.12 | 17.46 | — | |
| VanillaRetriever=Qwen3-8B, +S (Retriever)=false, Generator=Qwen2.5-14B, +S (Generator)=false, top-k=102025.12 | 15.87 | — | |
| MiA (Gen-Only)Retriever=Qwen3-8B, +S (Retriever)=false, Generator=Qwen2.5-14B, +S (Generator)=true, top-k=52025.12 | 15.87 | — | |
| VanillaRetriever=Qwen3-8B, +S (Retriever)=false, Generator=Qwen2.5-14B, +S (Generator)=false, top-k=52025.12 | 11.11 | — | |
| ComoRAGBase LLM=GPT-4o, Type=Multi-step RAG, Working Memory=true2025.12 | — | 63.49 | |
| ComoRAGBase LLM=Qwen2.5-32B-Instruct, Type=Multi-step RAG, Working Memory=true2025.12 | — | 57.6 | |
| DeepRAGBase LLM=GPT-4o, Type=Multi-step RAG, Working Memory=true2025.12 | — | 67.46 | |
| DeepRAGBase LLM=Qwen2.5-32B-Instruct, Type=Multi-step RAG, Working Memory=true2025.12 | — | 66.4 | |
| GraphRAGBase LLM=GPT-4o, Type=Traditional RAG, Working Memory=false2025.12 | — | 70.63 | |
| GraphRAGBase LLM=Qwen2.5-32B-Instruct, Type=Traditional RAG, Working Memory=false2025.12 | — | 62.7 | |
| HGMEMBase LLM=GPT-4o, Type=Ours, Working Memory=true2025.12 | — | 73.81 | |
| HGMEMBase LLM=Qwen2.5-32B-Instruct, Type=Ours, Working Memory=true2025.12 | — | 70.63 | |
| HippoRAG v2Base LLM=GPT-4o, Type=Traditional RAG, Working Memory=false2025.12 | — | 72.22 | |
| HippoRAG v2Base LLM=Qwen2.5-32B-Instruct, Type=Traditional RAG, Working Memory=false2025.12 | — | 68.25 | |
| LightRAGBase LLM=GPT-4o, Type=Traditional RAG, Working Memory=false2025.12 | — | 71.43 | |
| LightRAGBase LLM=Qwen2.5-32B-Instruct, Type=Traditional RAG, Working Memory=false2025.12 | — | 59.52 | |
| NaiveRAGBase LLM=GPT-4o, Type=Traditional RAG, Working Memory=false2025.12 | — | 67.46 | |
| NaiveRAGBase LLM=Qwen2.5-32B-Instruct, Type=Traditional RAG, Working Memory=false2025.12 | — | 64.29 |