Retrieval on STaRK-Prime 1.0 (test)
69.72Hit@1Regular RAG with Relation-grouped graph representation (Scenario 2)
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| Regular RAG with Relation-grouped graph representation (Scenario 2)RAG Paradigm=Regular RAG, Scenario Description=Scenario 2: RAG with Entity Description and Relations Documents, LLM Processing=with LLM answer generation and entity selection, Representation=Relation-grouped graph representation2026.06 | 69.72 | 82.57 | 67.28 | 75.31 | |
| Autonomous Agentic RAG (Scenario 8)RAG Paradigm=Autonomous Agentic RAG, Scenario Description=Scenario 8: Autonomous Agentic RAG with minimal tools, Retrieval Tools=doc retriever2026.06 | 68.81 | 84.4 | 77.45 | 75.49 | |
| GraphRAG (Scenario 5)RAG Paradigm=GraphRAG, Scenario Description=Scenario 5: GraphRAG with vector search and predefined KG, Max Paths=100, Number of Entity Subgraphs=202026.06 | 65.14 | 77.98 | 61.31 | 70.72 | |
| Agentic RAG (Scenario 7)RAG Paradigm=Agentic RAG, Scenario Description=Scenario 7: Agentic RAG with modular components as tools, Retrieval Tools=query reformatter, doc retriever, reranker2026.06 | 65.14 | 81.65 | 78.41 | 72.35 | |
| GraphRAG (Scenario 5)RAG Paradigm=GraphRAG, Scenario Description=Scenario 5: GraphRAG with vector search and predefined KG, Max Paths=100, Number of Entity Subgraphs=52026.06 | 64.22 | 77.98 | 63.53 | 70.11 | |
| Autonomous Agentic RAG (Scenario 8)RAG Paradigm=Autonomous Agentic RAG, Scenario Description=Scenario 8: Autonomous Agentic RAG with minimal tools, Retrieval Tools=doc retriever, Thinking Mode=enabled2026.06 | 62.96 | 80.56 | 76.43 | 71.6 | |
| GraphRAG (Scenario 5)RAG Paradigm=GraphRAG, Scenario Description=Scenario 5: GraphRAG with vector search and predefined KG, Max Paths=50, Number of Entity Subgraphs=52026.06 | 62.39 | 76.15 | 60.61 | 68.58 | |
| Modular RAG (Scenario 6)RAG Paradigm=Modular RAG, Scenario Description=Scenario 6: Modular RAG with well-defined workflow, Modular Components=query reformatter, doc retriever, reranker2026.06 | 62.39 | 80.73 | 78.57 | 70.3 | |
| Regular RAG (Scenario 1)RAG Paradigm=Regular RAG, Scenario Description=Scenario 1: RAG with Entity Description Documents only, LLM Processing=with LLM answer generation and entity selection2026.06 | 61.47 | 75.23 | 61.19 | 67.69 | |
| GraphRAG (Scenario 5)RAG Paradigm=GraphRAG, Scenario Description=Scenario 5: GraphRAG with vector search and predefined KG, Max Paths=200, Number of Entity Subgraphs=52026.06 | 61.47 | 76.15 | 61.77 | 68.31 | |
| Modular RAG (Scenario 6)RAG Paradigm=Modular RAG, Scenario Description=Scenario 6: Modular RAG with well-defined workflow, Modular Components=doc retriever only2026.06 | 61.47 | 75.23 | 61.19 | 67.69 | |
| Autonomous Agentic RAG (Scenario 9)RAG Paradigm=Autonomous Agentic RAG, Scenario Description=Scenario 9: Autonomous Agentic RAG with KG retrieval tools, Retrieval Tools=doc retriever, pre-defined KG retriever2026.06 | 60.55 | 78.9 | 71.25 | 68.34 | |
| Modular RAG (Scenario 6)RAG Paradigm=Modular RAG, Scenario Description=Scenario 6: Modular RAG with well-defined workflow, Modular Components=doc retriever, reranker2026.06 | 59.63 | 77.06 | 71.4 | 67.49 | |
| Regular RAG with Relation-grouped graph representation (Scenario 2)RAG Paradigm=Regular RAG, Scenario Description=Scenario 2: RAG with Entity Description and Relations Documents, LLM Processing=retrieval only (no LLM answer generation and entity selection), Representation=Relation-grouped graph representation2026.06 | 55.96 | 77.98 | 78.74 | 66.47 | |
| GraphRAG (Scenario 4)RAG Paradigm=GraphRAG, Scenario Description=Scenario 4: GraphRAG with vector search and computed KG from entity documents, LLM Processing=with LLM answer generation and entity selection2026.06 | 55.96 | 69.72 | 56.25 | 61.62 | |
| Regular RAG (Scenario 1)RAG Paradigm=Regular RAG, Scenario Description=Scenario 1: RAG with Entity Description Documents only, LLM Processing=retrieval only (no LLM answer generation and entity selection)2026.06 | 44.04 | 66.97 | 71.54 | 54.55 | |
| GraphRAG (Scenario 3)RAG Paradigm=GraphRAG, Scenario Description=Scenario 3: GraphRAG with predefined KG only, Max Paths=1002026.06 | 13.76 | 18.35 | 14.67 | 15.42 | |
| GraphRAG (Scenario 4)RAG Paradigm=GraphRAG, Scenario Description=Scenario 4: GraphRAG with vector search and computed KG from entity documents, LLM Processing=retrieval only (no LLM answer generation and entity selection)2026.06 | 0 | 14.68 | 63.4 | 10.36 |