Question Answering on NQ (EM, F1, Time, Rate)
67EMGraphRAG
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| GraphRAGBackbone=Mistral-7B-Instruct-v0.32026.04 | 67 | — | — | — | |
| KVIBackbone=Mistral-7B-Instruct-v0.32026.04 | 67 | — | — | — | |
| RAGBackbone=Mistral-7B-Instruct-v0.32026.04 | 63 | — | — | — | |
| GreedyModel=Qwen2.5-7B2026.06 | 60.35 | 70.87 | — | — | |
| AdaCADModel=Qwen2.5-7B2026.06 | 57.21 | 68.71 | — | — | |
| COIECDModel=Qwen2.5-7B, lambda=0.25, inner alpha=12026.06 | 53.22 | 65.81 | — | — | |
| Ψ-RAG øType=Tree-RAG, Reranker=Qwen3-Reranker-8B (Q3R), Abstract Type=keyword2026.05 | 50.6 | 63.58 | — | — | |
| TaskFT-SPEED-28+BoSFine-tuning=Task-adaptive, K=28, BoS=true2026.05 | 50.2 | 62.6 | — | — | |
| DPR + IRCoT + Q3RType=Traditional RAG, Reranker=Qwen3-Reranker-8B (Q3R), Iterative Retrieval Protocol=IRCoT2026.05 | 50.1 | 62.71 | — | — | |
| Ψ-RAG ⍋Type=Tree-RAG, Reranker=Qwen3-Reranker-8B (Q3R), Abstract Type=summative2026.05 | 50 | 63.45 | — | — | |
| TaskFT-FullFine-tuning=Task-adaptive, Visibility Policy=Full2026.05 | 49.3 | 62.2 | — | — | |
| TaskFT-SPEED-24+BoSFine-tuning=Task-adaptive, K=24, BoS=true2026.05 | 49.2 | 61.4 | — | — | |
| KVIBackbone=Qwen2.5-7B-Instruct2026.04 | 49 | — | — | — | |
| Full-depth LoRAAdaptation strategy=Full-depth LoRA2026.05 | 48.5 | 62.4 | — | — | |
| ARRModel=Qwen2.5-7B2026.06 | 48.38 | 61.2 | — | — | |
| BM25 + DPR + Q3RType=Traditional RAG, Reranker=Qwen3-Reranker-8B (Q3R)2026.05 | 48.2 | 60.58 | — | — | |
| TaskFT-SPEED-20+BoSFine-tuning=Task-adaptive, K=20, BoS=true2026.05 | 48.2 | 60.7 | — | — | |
| OffShelf-FT-SPEED+BoS-28Adaptation strategy=SPEED+BoS, K=282026.05 | 47.9 | 61.5 | — | — | |
| AdaCADModel=Llama2-13B2026.06 | 47.8 | 60.86 | — | — | |
| HippoRAG 2Type=Graph-RAG2026.05 | 47.4 | 62.42 | — | — | |
| GreedyModel=Llama2-13B2026.06 | 46.54 | 59.6 | — | — | |
| OffShelf-FT-SPEED+BoS-24Adaptation strategy=SPEED+BoS, K=242026.05 | 46.4 | 59.8 | — | — | |
| DPRType=Traditional RAG2026.05 | 46 | 59.09 | — | — | |
| Llama3.1 8B InstructAdaptation strategy=None2026.05 | 45.8 | 61.1 | — | — | |
| HippoRAG 2 + Q3RType=Graph-RAG, Reranker=Qwen3-Reranker-8B (Q3R)2026.05 | 45.7 | 60.99 | — | — | |
| BM25Type=Traditional RAG2026.05 | 45.4 | 56.76 | — | — | |
| OffShelf-FT-SPEED+BoS-20Adaptation strategy=SPEED+BoS, K=202026.05 | 45.4 | 58.7 | — | — | |
| COIECDModel=Llama2-13B, lambda=0.25, inner alpha=12026.06 | 44.94 | 60.42 | — | — | |
| GraphRAGBackbone=Qwen2.5-7B-Instruct2026.04 | 44 | — | — | — | |
| RAPTOR + Q3RType=Tree-RAG, Reranker=Qwen3-Reranker-8B (Q3R)2026.05 | 43.5 | 55.4 | — | — | |
| ARRModel=Llama2-13B2026.06 | 43.43 | 57.19 | — | — | |
| CoCoAModel=Llama2-13B, Renyi order=0.5, peakedness weight=5.0, entropy-gap weight=1.02026.06 | 43.02 | 58.36 | — | — | |
| RAGBackbone=Qwen2.5-7B-Instruct2026.04 | 41 | — | — | — | |
| GreedyModel=Mistral-7B2026.06 | 39.25 | 50.31 | — | — | |
| OffShelf-FT-SPEED+BoS-16Adaptation strategy=SPEED+BoS, K=162026.05 | 39.2 | 52.8 | — | — | |
| RawReader=Llama-3.3-70B Instruct, Retrieved chunks=Top-202026.03 | 39.1 | 53.4 | — | 100 | |
| RawReader=Llama-3.1-8B Instruct, Retrieved chunks=Top-202026.03 | 37.2 | 49.3 | — | 100 | |
| ARRModel=Mistral-7B2026.06 | 36.9 | 49.36 | — | — | |
| AdaCADModel=Mistral-7B2026.06 | 36.86 | 46.52 | — | — | |
| RawReader=Llama-3.3-70B Instruct, Retrieved chunks=Top-52026.03 | 36.8 | 51.1 | — | 100 | |
| ARRModel=Llama3-8B2026.06 | 36.37 | 51.58 | — | — | |
| LooCompReader=Llama-3.3-70B Instruct, Retrieved chunks=Top-202026.03 | 36.3 | 50.5 | 0.153 | 13.8 | |
| GraphRAGType=Graph-RAG2026.05 | 36 | 50.85 | — | — | |
| COIECDModel=Mistral-7B, lambda=0.25, inner alpha=12026.06 | 35.98 | 46.01 | — | — | |
| RawReader=Llama-3.1-8B Instruct, Retrieved chunks=Top-52026.03 | 35.4 | 47.3 | — | 100 | |
| LooCompReader=Llama-3.3-70B Instruct, Retrieved chunks=Top-52026.03 | 35.1 | 48.8 | 0.036 | 20 | |
| LooCompReader=Llama-3.1-8B Instruct, Retrieved chunks=Top-202026.03 | 34.4 | 46.5 | 0.153 | 13.8 | |
| LLMBackbone=Mistral-7B-Instruct-v0.32026.04 | 34 | — | — | — | |
| EXITReader=Llama-3.1-8B Instruct, Retrieved chunks=Top-52026.03 | 33.8 | 45 | 0.359 | 56.1 | |
| RECOMP-extReader=Llama-3.1-8B Instruct, Retrieved chunks=Top-52026.03 | 33.8 | 45.1 | 0.009 | 47.4 | |
| CoCoAModel=Qwen2.5-7B, Renyi order=0.5, peakedness weight=5.0, entropy-gap weight=1.02026.06 | 33.72 | 50.36 | — | — | |
| RECOMP-absReader=Llama-3.1-8B Instruct, Retrieved chunks=Top-52026.03 | 33.5 | 45.1 | 0.765 | 7.8 | |
| ProvenceReader=Llama-3.1-8B Instruct, Retrieved chunks=Top-52026.03 | 33.4 | 44.8 | 0.059 | 26.4 | |
| LooCompReader=Llama-3.1-8B Instruct, Retrieved chunks=Top-52026.03 | 33.4 | 44.9 | 0.036 | 20 | |
| RefinerReader=Llama-3.1-8B Instruct, Retrieved chunks=Top-52026.03 | 33.2 | 44.9 | 2.403 | 15.3 | |
| GreedyModel=Llama3-8B2026.06 | 31.61 | 47.68 | — | — | |
| CompActReader=Llama-3.1-8B Instruct, Retrieved chunks=Top-52026.03 | 31.4 | 42 | 2.573 | 12.1 | |
| CADModel=Llama2-13B, alpha=12026.06 | 30.41 | 48.12 | — | — | |
| LongLLMLinReader=Llama-3.1-8B Instruct, Retrieved chunks=Top-52026.03 | 29.8 | 41.2 | 0.345 | 46.2 | |
| AdaCADModel=Llama3-8B2026.06 | 28.33 | 44.93 | — | — | |
| COIECDModel=Llama3-8B, lambda=0.25, inner alpha=12026.06 | 22.26 | 39.81 | — | — | |
| CoCoAModel=Llama3-8B, Renyi order=0.5, peakedness weight=5.0, entropy-gap weight=1.02026.06 | 22.23 | 38.99 | — | — | |
| CADModel=Qwen2.5-7B, alpha=12026.06 | 21.85 | 32.99 | — | — | |
| LLMBackbone=Qwen2.5-7B-Instruct2026.04 | 20 | — | — | — | |
| KV PrefixBackbone=Mistral-7B-Instruct-v0.32026.04 | 17 | — | — | — | |
| CADModel=Mistral-7B, alpha=12026.06 | 15.84 | 22.76 | — | — | |
| CADModel=Llama3-8B, alpha=12026.06 | 14.76 | 30.74 | — | — | |
| KV PrefixBackbone=Qwen2.5-7B-Instruct2026.04 | 14 | — | — | — | |
| CoCoAModel=Mistral-7B, Renyi order=0.5, peakedness weight=5.0, entropy-gap weight=1.02026.06 | 13.86 | 29.24 | — | — |