Question Answering on HotpotQA (Average Performance Score)
78.4Average Performance ScoreDeepSeek-R1 w/ GuarantRAG
Evaluation Results
| Method | Links | |
|---|---|---|
| DeepSeek-R1 w/ GuarantRAGBackbone=DeepSeek-R1, RAG Strategy=GuarantRAG, Retriever=BM252026.04 | 78.4 | |
| DeepSeek-R1 w/ P-RAGBackbone=DeepSeek-R1, RAG Strategy=P-RAG, Retriever=BM252026.04 | 76.9 | |
| DeepSeek-V3 w/ GuarantRAGBackbone=DeepSeek-V3, RAG Strategy=GuarantRAG, Retriever=BM252026.04 | 76.8 | |
| Gemini-2.5-Pro w/ GuarantRAGBackbone=Gemini-2.5-Pro, RAG Strategy=GuarantRAG, Retriever=BM252026.04 | 75.6 | |
| DeepSeek-R1 w/ SelfRAGBackbone=DeepSeek-R1, RAG Strategy=SelfRAG, Retriever=BM252026.04 | 75.3 | |
| DeepSeek-V3 w/ P-RAGBackbone=DeepSeek-V3, RAG Strategy=P-RAG, Retriever=BM252026.04 | 75.2 | |
| Claude-3.7-Sonnet w/ GuarantRAGBackbone=Claude-3.7-Sonnet, RAG Strategy=GuarantRAG, Retriever=BM252026.04 | 74.3 | |
| Gemini-2.5-Pro w/ P-RAGBackbone=Gemini-2.5-Pro, RAG Strategy=P-RAG, Retriever=BM252026.04 | 74.1 | |
| DeepSeek-V3 w/ SelfRAGBackbone=DeepSeek-V3, RAG Strategy=SelfRAG, Retriever=BM252026.04 | 73.8 | |
| DeepSeek-R1 w/ Standard RAGBackbone=DeepSeek-R1, RAG Strategy=Standard RAG, Retriever=BM252026.04 | 73.7 | |
| Claude-3.7-Sonnet w/ P-RAGBackbone=Claude-3.7-Sonnet, RAG Strategy=P-RAG, Retriever=BM252026.04 | 72.8 | |
| Gemini-2.5-Pro w/ SelfRAGBackbone=Gemini-2.5-Pro, RAG Strategy=SelfRAG, Retriever=BM252026.04 | 72.4 | |
| DeepSeek-V3 w/ Standard RAGBackbone=DeepSeek-V3, RAG Strategy=Standard RAG, Retriever=BM252026.04 | 72.1 | |
| DeepSeek-R1Backbone=DeepSeek-R1, RAG Strategy=Base, Retriever=BM252026.04 | 71.2 | |
| Claude-3.7-Sonnet w/ SelfRAGBackbone=Claude-3.7-Sonnet, RAG Strategy=SelfRAG, Retriever=BM252026.04 | 71.1 | |
| Gemini-2.5-Pro w/ Standard RAGBackbone=Gemini-2.5-Pro, RAG Strategy=Standard RAG, Retriever=BM252026.04 | 70.8 | |
| Claude-3.7-Sonnet w/ Standard RAGBackbone=Claude-3.7-Sonnet, RAG Strategy=Standard RAG, Retriever=BM252026.04 | 69.5 | |
| DeepSeek-V3Backbone=DeepSeek-V3, RAG Strategy=Base, Retriever=BM252026.04 | 69.4 | |
| Gemini-2.5-ProBackbone=Gemini-2.5-Pro, RAG Strategy=Base, Retriever=BM252026.04 | 68.1 | |
| Claude-3.7-SonnetBackbone=Claude-3.7-Sonnet, RAG Strategy=Base, Retriever=BM252026.04 | 66.7 |