Question Answering on MetaQA (Hop-based EM@1 Evaluation)
92.71-hop EM@1CLAUSE
Evaluation Results
| Method | Links | |||
|---|---|---|---|---|
| CLAUSEFamily=Ours, LLM Reader=LLaMA3.3-70B2025.09 | 92.7 | 87.7 | 87.2 | |
| CLAUSEFamily=Ours, Reader Model=GPT-OSS-120B2025.09 | 92.2 | 88 | 87.5 | |
| CLAUSEFamily=Ours2025.09 | 91 | 87.3 | 85.5 | |
| KG-AgentFamily=Agent-based, Reader Model=GPT-OSS-120B2025.09 | 88.9 | 79.4 | 76.9 | |
| KG-AgentFamily=Agent-based, LLM Reader=LLaMA3.3-70B2025.09 | 88 | 78.8 | 76.3 | |
| KG-AgentFamily=Agent-based (Qwen3-32B)2025.09 | 87.3 | 78 | 75.4 | |
| AutoGenFamily=Agent-based, Reader Model=GPT-OSS-120B2025.09 | 87 | 66 | 55.8 | |
| AutoGenFamily=Agent-based, LLM Reader=LLaMA3.3-70B2025.09 | 86.2 | 56.5 | 55.1 | |
| AutoGenFamily=Agent-based (Qwen3-32B)2025.09 | 85.2 | 55.7 | 53.5 | |
| ReActFamily=Agent-based, Reader Model=GPT-OSS-120B2025.09 | 84.1 | 65.5 | 53.5 | |
| ReActFamily=Agent-based, LLM Reader=LLaMA3.3-70B2025.09 | 83 | 53.3 | 50.6 | |
| ReActFamily=Agent-based (Qwen3-32B)2025.09 | 82.3 | 52.1 | 49.4 | |
| Graph-of-ThoughtsFamily=Agent-based, Reader Model=GPT-OSS-120B2025.09 | 81.7 | 49.3 | 52.6 | |
| Graph-of-ThoughtsFamily=Agent-based, LLM Reader=LLaMA3.3-70B2025.09 | 81 | 49 | 47.6 | |
| Graph-of-ThoughtsFamily=Agent-based (Qwen3-32B)2025.09 | 79.5 | 48.4 | 46.3 | |
| GraphRAGFamily=RAG-based, Reader Model=GPT-OSS-120B2025.09 | 67.8 | 55.6 | 54.3 | |
| Hybrid RAGFamily=RAG-based, Reader Model=GPT-OSS-120B2025.09 | 67 | 49.2 | 52.8 | |
| GraphRAGFamily=RAG-based, LLM Reader=LLaMA3.3-70B2025.09 | 65 | 49.5 | 50.3 | |
| Vanilla RAGFamily=RAG-based, Reader Model=GPT-OSS-120B2025.09 | 64.8 | 44.9 | 52.5 | |
| Hybrid RAGFamily=RAG-based, LLM Reader=LLaMA3.3-70B2025.09 | 64.3 | 43 | 45.8 | |
| GraphRAGFamily=RAG-based (Qwen3-32B)2025.09 | 63.5 | 48 | 44.4 | |
| Hybrid RAGFamily=RAG-based (Qwen3-32B)2025.09 | 63 | 41.5 | 34.1 | |
| LightRAGFamily=RAG-based, Reader Model=GPT-OSS-120B2025.09 | 62.9 | 43 | 52.4 | |
| GPT-OSS-120BFamily=Pretrained-LLMs2025.09 | 62.7 | 41.5 | 52.3 | |
| Vanilla RAGFamily=RAG-based, LLM Reader=LLaMA3.3-70B2025.09 | 62 | 38.6 | 45.4 | |
| Vanilla RAGFamily=RAG-based (Qwen3-32B)2025.09 | 60.2 | 37.6 | 33 | |
| LightRAGFamily=RAG-based, LLM Reader=LLaMA3.3-70B2025.09 | 59.6 | 36.8 | 45 | |
| LLaMA3.3-70BFamily=Pretrained-LLMs2025.09 | 57.2 | 29 | 44.2 | |
| LightRAGFamily=RAG-based (Qwen3-32B)2025.09 | 54 | 35 | 32 | |
| Qwen3-32BFamily=Pretrained-LLMs2025.09 | 52.5 | 22.8 | 39 |