Retrieval-Augmented Generation on HotpotQA
51.8Reliability Score (RS)CDA-M
Evaluation results
| Method | Links | |||||||
|---|---|---|---|---|---|---|---|---|
| CDA-MBackbone=LLAMA3 8B INSTRUCT2024.12 | 51.8 | — | — | — | — | — | — | |
| CDABackbone=LLAMA3 8B INSTRUCT2024.12 | 51.77 | — | — | — | — | — | — | |
| FSBBackbone=LLAMA3 8B INSTRUCT2024.12 | 51.07 | — | — | — | — | — | — | |
| ABSTAINBackbone=LLAMA3 8B INSTRUCT2024.12 | 50.71 | — | — | — | — | — | — | |
| ACD-ABackbone=LLAMA3 8B INSTRUCT2024.12 | 50.35 | — | — | — | — | — | — | |
| ENTROPY (first-token)Backbone=LLAMA3 8B INSTRUCT2024.12 | 50.24 | — | — | — | — | — | — | |
| ENTROPY (max)Backbone=LLAMA3 8B INSTRUCT2024.12 | 48.34 | — | — | — | — | — | — | |
| ENTROPY (average)Backbone=LLAMA3 8B INSTRUCT2024.12 | 47.67 | — | — | — | — | — | — | |
| FSBBackbone=MISTRAL 7B INSTRUCT2024.12 | 47.32 | — | — | — | — | — | — | |
| ACD-ABackbone=MISTRAL 7B INSTRUCT2024.12 | 47.14 | — | — | — | — | — | — | |
| CDA-MBackbone=MISTRAL 7B INSTRUCT2024.12 | 47.13 | — | — | — | — | — | — | |
| ENTROPY (max)Backbone=MISTRAL 7B INSTRUCT2024.12 | 47.06 | — | — | — | — | — | — | |
| CDABackbone=MISTRAL 7B INSTRUCT2024.12 | 46.94 | — | — | — | — | — | — | |
| ENTROPY (first-token)Backbone=MISTRAL 7B INSTRUCT2024.12 | 46.82 | — | — | — | — | — | — | |
| ENTROPY (average)Backbone=MISTRAL 7B INSTRUCT2024.12 | 46.69 | — | — | — | — | — | — | |
| ENTROPY (max)Backbone=LLAMA2 7B CHAT2024.12 | 45.69 | — | — | — | — | — | — | |
| FSBBackbone=LLAMA2 13B CHAT2024.12 | 45.28 | — | — | — | — | — | — | |
| CDABackbone=LLAMA2 7B CHAT2024.12 | 45.07 | — | — | — | — | — | — | |
| CDA-MBackbone=LLAMA2 7B CHAT2024.12 | 45.02 | — | — | — | — | — | — | |
| ENTROPY (average)Backbone=LLAMA2 7B CHAT2024.12 | 45.01 | — | — | — | — | — | — | |
| CDA-MBackbone=LLAMA2 13B CHAT2024.12 | 44.73 | — | — | — | — | — | — | |
| ENTROPY (first-token)Backbone=LLAMA2 7B CHAT2024.12 | 44.71 | — | — | — | — | — | — | |
| ENTROPY (first-token)Backbone=LLAMA2 13B CHAT2024.12 | 44.31 | — | — | — | — | — | — | |
| CDABackbone=LLAMA2 13B CHAT2024.12 | 44.06 | — | — | — | — | — | — | |
| ENTROPY (max)Backbone=LLAMA2 13B CHAT2024.12 | 43.93 | — | — | — | — | — | — | |
| ENTROPY (min)Backbone=LLAMA3 8B INSTRUCT2024.12 | 43.58 | — | — | — | — | — | — | |
| ENTROPY (average)Backbone=LLAMA2 13B CHAT2024.12 | 43.31 | — | — | — | — | — | — | |
| ENTROPY (min)Backbone=MISTRAL 7B INSTRUCT2024.12 | 42.76 | — | — | — | — | — | — | |
| ABSTAINBackbone=MISTRAL 7B INSTRUCT2024.12 | 41.99 | — | — | — | — | — | — | |
| ENTROPY (min)Backbone=LLAMA2 7B CHAT2024.12 | 41.65 | — | — | — | — | — | — | |
| ACD-ABackbone=LLAMA2 13B CHAT2024.12 | 41.62 | — | — | — | — | — | — | |
| ENTROPY (min)Backbone=LLAMA2 13B CHAT2024.12 | 39.88 | — | — | — | — | — | — | |
| SELF-ASKBackbone=LLAMA3 8B INSTRUCT2024.12 | 39.61 | — | — | — | — | — | — | |
| FSBBackbone=LLAMA2 7B CHAT2024.12 | 39.58 | — | — | — | — | — | — | |
| SELF-ASKBackbone=MISTRAL 7B INSTRUCT2024.12 | 38.78 | — | — | — | — | — | — | |
| ACD-ABackbone=LLAMA2 7B CHAT2024.12 | 35.05 | — | — | — | — | — | — | |
| ACDBackbone=LLAMA3 8B INSTRUCT2024.12 | 34.05 | — | — | — | — | — | — | |
| SELF-ASKBackbone=LLAMA2 7B CHAT2024.12 | 33.99 | — | — | — | — | — | — | |
| SELF-ASKBackbone=LLAMA2 13B CHAT2024.12 | 33.68 | — | — | — | — | — | — | |
| ACDBackbone=LLAMA2 13B CHAT2024.12 | 32.73 | — | — | — | — | — | — | |
| CONTEXTBackbone=LLAMA3 8B INSTRUCT2024.12 | 31 | — | — | — | — | — | — | |
| CONTEXTBackbone=LLAMA2 13B CHAT2024.12 | 30.57 | — | — | — | — | — | — | |
| ACDBackbone=LLAMA2 7B CHAT2024.12 | 30.04 | — | — | — | — | — | — | |
| CONTEXTBackbone=MISTRAL 7B INSTRUCT2024.12 | 29 | — | — | — | — | — | — | |
| ACDBackbone=MISTRAL 7B INSTRUCT2024.12 | 28.91 | — | — | — | — | — | — | |
| CONTEXTBackbone=LLAMA2 7B CHAT2024.12 | 28.28 | — | — | — | — | — | — | |
| CADBackbone=LLAMA3 8B INSTRUCT2024.12 | 28.24 | — | — | — | — | — | — | |
| CADBackbone=LLAMA2 13B CHAT2024.12 | 27.59 | — | — | — | — | — | — | |
| CADBackbone=MISTRAL 7B INSTRUCT2024.12 | 26.08 | — | — | — | — | — | — | |
| CADBackbone=LLAMA2 7B CHAT2024.12 | 25.86 | — | — | — | — | — | — | |
| ABSTAINBackbone=LLAMA2 13B CHAT2024.12 | 24.54 | — | — | — | — | — | — | |
| ABSTAINBackbone=LLAMA2 7B CHAT2024.12 | 20.02 | — | — | — | — | — | — | |
| ConsJudgeGenerator=MiniCPM-2.4B, Reward Model Backbone=Llama3-8B-Instruct2025.02 | — | 30.45 | — | — | — | — | — | |
| ConsJudgeGenerator=MiniCPM-2.4B, Reward Model Backbone=Qwen2.5-14B-Instruct2025.02 | — | 30.84 | — | — | — | — | — | |
| ConsJudgeGenerator=Llama3-8B-Instruct, Reward Model Backbone=Llama3-8B-Instruct2025.02 | — | 37.54 | — | — | — | — | — | |
| Graph-R1 (ours)Backbone=Qwen2.5-7B-Instruct, Knowledge Representation=Graph-based, Learning Strategy=Training2025.07 | — | — | — | — | — | 62.69 | 80.03 | |
| GroundedCacheVariant=no-support2026.05 | — | — | 0.827 | 1.37 | 18.2 | — | — | |
| GroundedCacheVariant=full2026.05 | — | — | 1.058 | 1.07 | 6.3 | — | — | |
| mtRAGVariant=no-cache (always generate)2026.05 | — | — | 1.128 | 1 | 0 | — | — | |
| Naive CachingVariant=naive2026.05 | — | — | 0.651 | 1.73 | 26.1 | — | — | |
| NaiveGenerationBackbone=GPT-4o-mini, Knowledge Interaction=No, Learning Strategy=Prompting2025.07 | — | — | — | — | — | 31.79 | 78.48 | |
| Raw MetricGenerator=MiniCPM-2.4B, Reward Model=Raw Metric2025.02 | — | 30.09 | — | — | — | — | — | |
| Raw MetricGenerator=Llama3-8B-Instruct, Reward Model=Raw Metric2025.02 | — | 36.95 | — | — | — | — | — | |
| SFTGenerator=MiniCPM-2.4B, Reward Model Backbone=Llama3-8B-Instruct2025.02 | — | 29.55 | — | — | — | — | — | |
| SFTGenerator=MiniCPM-2.4B, Reward Model Backbone=Qwen2.5-14B-Instruct2025.02 | — | 27.64 | — | — | — | — | — | |
| SFTGenerator=Llama3-8B-Instruct, Reward Model Backbone=Llama3-8B-Instruct2025.02 | — | 35.95 | — | — | — | — | — | |
| StandardRAGBackbone=GPT-4o-mini, Knowledge Representation=Chunk-based, Learning Strategy=Prompting2025.07 | — | — | — | — | — | 46.7 | 81.88 | |
| Vanilla LLMGenerator=MiniCPM-2.4B, Reward Model Backbone=Llama3-8B-Instruct2025.02 | — | 29.64 | — | — | — | — | — | |
| Vanilla LLMGenerator=MiniCPM-2.4B, Reward Model Backbone=Qwen2.5-14B-Instruct2025.02 | — | 28.96 | — | — | — | — | — | |
| Vanilla LLMGenerator=Llama3-8B-Instruct, Reward Model Backbone=Llama3-8B-Instruct2025.02 | — | 32.84 | — | — | — | — | — |