Question Answering on NQ, TruthfulQA, WoW, HotpotQA, and ELI5
82.9Avg Score (All Datasets)DeepSeek-R1 w/ GuarantRAG
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| DeepSeek-R1 w/ GuarantRAGBackbone=DeepSeek-R1, RAG Strategy=GuarantRAG, Retriever=BM252026.04 | 82.9 | 8.4 | 91.3 | 4.71 | |
| DeepSeek-V3 w/ GuarantRAGBackbone=DeepSeek-V3, RAG Strategy=GuarantRAG, Retriever=BM252026.04 | 81.4 | 9.6 | 90.4 | 4.68 | |
| DeepSeek-R1 w/ P-RAGBackbone=DeepSeek-R1, RAG Strategy=P-RAG, Retriever=BM252026.04 | 81.3 | 10.2 | 89.5 | 4.64 | |
| Gemini-2.5-Pro w/ GuarantRAGBackbone=Gemini-2.5-Pro, RAG Strategy=GuarantRAG, Retriever=BM252026.04 | 80.5 | 10.3 | 89.6 | 4.66 | |
| DeepSeek-V3 w/ P-RAGBackbone=DeepSeek-V3, RAG Strategy=P-RAG, Retriever=BM252026.04 | 79.9 | 11.4 | 88.7 | 4.61 | |
| DeepSeek-R1 w/ SelfRAGBackbone=DeepSeek-R1, RAG Strategy=SelfRAG, Retriever=BM252026.04 | 79.7 | 11.8 | 87.9 | 4.57 | |
| Claude-3.7-Sonnet w/ GuarantRAGBackbone=Claude-3.7-Sonnet, RAG Strategy=GuarantRAG, Retriever=BM252026.04 | 79.3 | 11.4 | 88.9 | 4.63 | |
| Gemini-2.5-Pro w/ P-RAGBackbone=Gemini-2.5-Pro, RAG Strategy=P-RAG, Retriever=BM252026.04 | 78.9 | 12 | 87.9 | 4.59 | |
| DeepSeek-V3 w/ SelfRAGBackbone=DeepSeek-V3, RAG Strategy=SelfRAG, Retriever=BM252026.04 | 78.5 | 12.9 | 87.1 | 4.54 | |
| DeepSeek-R1 w/ Standard RAGBackbone=DeepSeek-R1, RAG Strategy=Standard RAG, Retriever=BM252026.04 | 78.1 | 13.1 | 86.2 | 4.49 | |
| Claude-3.7-Sonnet w/ P-RAGBackbone=Claude-3.7-Sonnet, RAG Strategy=P-RAG, Retriever=BM252026.04 | 77.8 | 13.1 | 87.1 | 4.56 | |
| Gemini-2.5-Pro w/ SelfRAGBackbone=Gemini-2.5-Pro, RAG Strategy=SelfRAG, Retriever=BM252026.04 | 77.3 | 13.6 | 86.3 | 4.52 | |
| DeepSeek-V3 w/ Standard RAGBackbone=DeepSeek-V3, RAG Strategy=Standard RAG, Retriever=BM252026.04 | 76.8 | 14.2 | 85.3 | 4.47 | |
| Claude-3.7-Sonnet w/ SelfRAGBackbone=Claude-3.7-Sonnet, RAG Strategy=SelfRAG, Retriever=BM252026.04 | 76.1 | 14.7 | 85.4 | 4.49 | |
| DeepSeek-R1Backbone=DeepSeek-R1, RAG Strategy=Base, Retriever=BM252026.04 | 75.7 | 15.3 | — | 4.84 | |
| Gemini-2.5-Pro w/ Standard RAGBackbone=Gemini-2.5-Pro, RAG Strategy=Standard RAG, Retriever=BM252026.04 | 75.7 | 15.1 | 84.6 | 4.44 | |
| Claude-3.7-Sonnet w/ Standard RAGBackbone=Claude-3.7-Sonnet, RAG Strategy=Standard RAG, Retriever=BM252026.04 | 74.5 | 16.2 | 83.8 | 4.42 | |
| DeepSeek-V3Backbone=DeepSeek-V3, RAG Strategy=Base, Retriever=BM252026.04 | 74.2 | 16.8 | — | 4.81 | |
| Gemini-2.5-ProBackbone=Gemini-2.5-Pro, RAG Strategy=Base, Retriever=BM252026.04 | 73.2 | 17.6 | — | 4.79 | |
| Claude-3.7-SonnetBackbone=Claude-3.7-Sonnet, RAG Strategy=Base, Retriever=BM252026.04 | 71.9 | 18.9 | — | 4.77 |