Question Answering on Natural Questions (EM, F1, LLM)
45.72EMLLM (with RAG*)
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| LLM (with RAG*)Backbone=Mistral-7B-Instruct, Top-k=5, Comp. Rate=-2026.01 | 45.72 | 54.49 | 55.73 | |
| SeleComBackbone=Mistral-7B-Instruct, Top-k=5, Comp. Rate=82x2026.01 | 44.27 | 52.85 | 54.2 | |
| LLM (with RAG*)Backbone=Qwen2.5-7B-Instruct, Top-k=5, Comp. Rate=-2026.01 | 42.76 | 51.94 | 54.03 | |
| SeleComBackbone=Qwen2.5-7B-Instruct, Top-k=5, Comp. Rate=82x2026.01 | 41.86 | 50.94 | 52.22 | |
| SeleComBackbone=Mistral-7B-Instruct, Top-k=1, Comp. Rate=82x2026.01 | 41.8 | 49.72 | 50.91 | |
| LLM (with RAG*)Backbone=Mistral-7B-Instruct, Top-k=1, Comp. Rate=-2026.01 | 41.12 | 49.86 | 50.88 | |
| LLM (with RAG*)Backbone=Qwen2.5-7B-Instruct, Top-k=1, Comp. Rate=-2026.01 | 39.31 | 48.15 | 49.91 | |
| SeleComBackbone=Qwen2.5-7B-Instruct, Top-k=1, Comp. Rate=82x2026.01 | 38.58 | 47.02 | 49.02 | |
| COCOMBackbone=Mistral-7B-Instruct, Top-k=1, Comp. Rate=128x2026.01 | 31.86 | 40.15 | 42.44 | |
| LLM (with RAG)Backbone=Qwen2.5-7B-Instruct, Top-k=1, Comp. Rate=-2026.01 | 15.01 | 24.96 | 38.92 | |
| ICAEBackbone=Mistral-7B-Instruct, Top-k=1, Comp. Rate=4x2026.01 | 14.43 | 25.99 | 37.83 | |
| RECOMPBackbone=Mistral-7B-Instruct, Top-k=1, Comp. Rate=N/A2026.01 | 8.14 | 19.06 | 41.02 | |
| LLM (with RAG)Backbone=Mistral-7B-Instruct, Top-k=1, Comp. Rate=-2026.01 | 7.98 | 18.19 | 40.95 | |
| LLM (w/o RAG)Backbone=Qwen2.5-7B-Instruct, Top-k=1, Comp. Rate=-2026.01 | 7.78 | 17.18 | 31.51 | |
| LLMLingua-2Backbone=Mistral-7B-Instruct, Top-k=1, Comp. Rate=4x2026.01 | 5.87 | 15.13 | 35.29 | |
| LLM (w/o RAG)Backbone=Mistral-7B-Instruct, Top-k=1, Comp. Rate=-2026.01 | 4.13 | 14.3 | 38.21 | |
| xRAGBackbone=Mistral-7B-Instruct, Top-k=1, Comp. Rate=164x2026.01 | 3.02 | 12.45 | 40.83 | |
| PISCOBackbone=Mistral-7B-Instruct, Top-k=1, Comp. Rate=64x2026.01 | 0.06 | 9.49 | 39.87 |