Open-Domain Question Answering on NaturalQuestions (NQ)
52.88SubEMStable-RAG
Evaluation results
| Method | Links | ||
|---|---|---|---|
| Stable-RAGBackbone=LLaMA3-8B-Instruct, Retriever=DPR, exhaustive decoding=true2026.01 | 52.88 | 51.78 | |
| Stable-RAGBackbone=Qwen3-8B, Retriever=DPR, exhaustive decoding=true2026.01 | 52.12 | 52.38 | |
| Stable-RAGBackbone=LLaMA3-8B-Instruct, Retriever=DPR, exhaustive decoding=false2026.01 | 52.02 | 50.72 | |
| Stable-RAGBackbone=Qwen3-8B, Retriever=DPR, exhaustive decoding=false2026.01 | 51.69 | 51.78 | |
| ATMBackbone=Qwen3-8B, Retriever=DPR2026.01 | 50.94 | 51.03 | |
| Pos2DistillBackbone=Qwen3-8B, Retriever=DPR2026.01 | 50.71 | 50.93 | |
| Vanilla RAGBackbone=Qwen3-8B, Retriever=DPR2026.01 | 50.55 | 50.67 | |
| RAATBackbone=Qwen3-8B, Retriever=DPR2026.01 | 50.12 | 50.03 | |
| Ms-PoEBackbone=Qwen3-8B, Retriever=DPR2026.01 | 50.04 | 50.08 | |
| ATMBackbone=LLaMA3-8B-Instruct, Retriever=DPR2026.01 | 49.78 | 50.19 | |
| RetRobustBackbone=Qwen3-8B, Retriever=DPR2026.01 | 49.5 | 50.81 | |
| Pos2DistillBackbone=LLaMA3-8B-Instruct, Retriever=DPR2026.01 | 49.25 | 48.37 | |
| RAATBackbone=LLaMA3-8B-Instruct, Retriever=DPR2026.01 | 49.12 | 49.85 | |
| Stable-RAGBackbone=LLaMA3-8B-Instruct, Retriever=Contriever, exhaustive decoding=true2026.01 | 48.75 | 46.58 | |
| RetRobustBackbone=LLaMA3-8B-Instruct, Retriever=DPR2026.01 | 48.7 | 49.29 | |
| Stable-RAGBackbone=LLaMA3-8B-Instruct, Retriever=Contriever, exhaustive decoding=false2026.01 | 48.14 | 45.8 | |
| Stable-RAGBackbone=Qwen3-8B, Retriever=Contriever, exhaustive decoding=true2026.01 | 46.94 | 47.13 | |
| Vanilla SFTBackbone=LLaMA3-8B-Instruct, Retriever=DPR2026.01 | 46.2 | 49.44 | |
| Stable-RAGBackbone=Qwen3-8B, Retriever=Contriever, exhaustive decoding=false2026.01 | 46.12 | 46.79 | |
| Vanilla RAGBackbone=LLaMA3-8B-Instruct, Retriever=DPR2026.01 | 45.81 | 47.8 | |
| Vanilla SFTBackbone=Qwen3-8B, Retriever=DPR2026.01 | 45.6 | 49.19 | |
| Ms-PoEBackbone=LLaMA3-8B-Instruct, Retriever=DPR2026.01 | 45.58 | 47.53 | |
| ATMBackbone=Qwen3-8B, Retriever=Contriever2026.01 | 45.47 | 45.86 | |
| RAATBackbone=Qwen3-8B, Retriever=Contriever2026.01 | 45.13 | 45.87 | |
| Pos2DistillBackbone=Qwen3-8B, Retriever=Contriever2026.01 | 44.89 | 45.52 | |
| Vanilla RAGBackbone=Qwen3-8B, Retriever=Contriever2026.01 | 44.65 | 45.34 | |
| Pos2DistillBackbone=LLaMA3-8B-Instruct, Retriever=Contriever2026.01 | 44.58 | 43.12 | |
| Ms-PoEBackbone=Qwen3-8B, Retriever=Contriever2026.01 | 44.39 | 45.12 | |
| ATMBackbone=LLaMA3-8B-Instruct, Retriever=Contriever2026.01 | 43.75 | 44.88 | |
| RetRobustBackbone=Qwen3-8B, Retriever=Contriever2026.01 | 43.1 | 44.99 | |
| RAATBackbone=LLaMA3-8B-Instruct, Retriever=Contriever2026.01 | 42.33 | 43.85 | |
| Vanilla SFTBackbone=LLaMA3-8B-Instruct, Retriever=Contriever2026.01 | 42.1 | 44.78 | |
| RetRobustBackbone=LLaMA3-8B-Instruct, Retriever=Contriever2026.01 | 41.82 | 44.26 | |
| Vanilla SFTBackbone=Qwen3-8B, Retriever=Contriever2026.01 | 41.41 | 45.05 | |
| Vanilla RAGBackbone=LLaMA3-8B-Instruct, Retriever=Contriever2026.01 | 40.75 | 42.82 | |
| Ms-PoEBackbone=LLaMA3-8B-Instruct, Retriever=Contriever2026.01 | 40.32 | 42.49 | |
| Direct GenerationBackbone=LLaMA3-8B-Instruct, Retriever=Contriever2026.01 | 25.18 | 29.11 | |
| Direct GenerationBackbone=LLaMA3-8B-Instruct, Retriever=DPR2026.01 | 25.18 | 29.11 | |
| Naive GenerationBackbone=Qwen3-8B, Retriever=Contriever2026.01 | 21.94 | 24.07 | |
| Naive GenerationBackbone=Qwen3-8B, Retriever=DPR2026.01 | 21.94 | 24.07 |