Question Answering on NarrativeQA
21.5BLEU-1DTCRS (w/o global)
Evaluation results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| DTCRS (w/o global)Backbone LLM=GPT-4o-mini2026.04 | 21.5 | 26.9 | 1.3 | 14.7 | — | |
| DPRBackbone LLM=GPT-4o-mini2026.04 | 21.4 | 26.5 | 1.3 | 14.4 | — | |
| DTCRS (w/o classify)Backbone LLM=GPT-4o-mini2026.04 | 21.4 | 26.2 | 1.2 | 15.1 | — | |
| DTCRS (w/o contents)Backbone LLM=GPT-4o-mini2026.04 | 21.4 | 26.7 | 1.1 | 14.3 | — | |
| RAPTORBackbone LLM=GPT-4o-mini2026.04 | 21.2 | 25 | 1.1 | 14.1 | — | |
| DTCRSBackbone LLM=GPT-4o-mini2026.04 | 21.1 | 26 | 1.3 | 14.2 | — | |
| DTCRSBackbone LLM=DeepSeek-V2-Lite-Chat2026.04 | 19.1 | 28.1 | 0.9 | 20.1 | — | |
| DTCRS (w/o global)Backbone LLM=DeepSeek-V2-Lite-Chat2026.04 | 18.9 | 27.9 | 1.3 | 19.7 | — | |
| RAPTORBackbone LLM=DeepSeek-V2-Lite-Chat2026.04 | 17.9 | 25.4 | 1 | 18.3 | — | |
| DTCRS (w/o classify)Backbone LLM=DeepSeek-V2-Lite-Chat2026.04 | 17.3 | 25.3 | 0.7 | 17.7 | — | |
| DPRBackbone LLM=DeepSeek-V2-Lite-Chat2026.04 | 15.2 | 21.3 | 0.5 | 15 | — | |
| DTCRS (w/o contents)Backbone LLM=DeepSeek-V2-Lite-Chat2026.04 | 15.2 | 21.4 | 0.6 | 15.2 | — | |
| ArchRAGBaseline Type=Our proposed2025.02 | 11.5 | — | — | 15.6 | 17.6 | |
| Zero-shotBaseline Type=Inference-only2025.02 | 8 | — | — | 7.9 | 8.6 | |
| RAPTORBaseline Type=Graph-based RAG2025.02 | 5.5 | — | — | 12.5 | 9.1 | |
| CoTBaseline Type=Inference-only2025.02 | 5 | — | — | 8.1 | 6.4 | |
| HyLightRAGBaseline Type=Graph-based RAG2025.02 | 5 | — | — | 9.4 | 7 | |
| LLightRAGBaseline Type=Graph-based RAG2025.02 | 4.5 | — | — | 8.7 | 6.6 | |
| HLightRAGBaseline Type=Graph-based RAG2025.02 | 4.4 | — | — | 8.1 | 6.1 | |
| LGraphRAGBaseline Type=Graph-based RAG2025.02 | 3.9 | — | — | 3.3 | 3.5 | |
| HippoRAGBaseline Type=Graph-based RAG2025.02 | 2.2 | — | — | 5 | 2.8 | |
| BM25Baseline Type=Retrieval-only2025.02 | 2 | — | — | 4.9 | 2.8 | |
| Vanilla RAGBaseline Type=Retrieval-only2025.02 | 2 | — | — | 4.9 | 2.8 | |
| Retriever + Reader2024.01 | 0.353 | 0.32 | 7.5 | 0.111 | — | |
| RAPTOR + UnifiedQABase Model=UnifiedQA 3B2024.01 | 0.235 | 0.308 | 6.4 | 0.191 | — | |
| Recursively Summarizing Books2024.01 | 0.223 | 0.216 | 4.2 | 0.106 | — | |
| BM25 + BERT2024.01 | 0.145 | 0.155 | 1.4 | 0.05 | — | |
| BiDAF2024.01 | 0.057 | 0.062 | 0.3 | 0.037 | — |