Question Answering on WoW
84.3Average Performance ScoreDeepSeek-R1 w/ GuarantRAG
Evaluation Results
| Method | Links | |
|---|---|---|
| DeepSeek-R1 w/ GuarantRAGBackbone=DeepSeek-R1, RAG Strategy=GuarantRAG, Retriever=BM252026.04 | 84.3 | |
| DeepSeek-V3 w/ GuarantRAGBackbone=DeepSeek-V3, RAG Strategy=GuarantRAG, Retriever=BM252026.04 | 83.2 | |
| DeepSeek-R1 w/ P-RAGBackbone=DeepSeek-R1, RAG Strategy=P-RAG, Retriever=BM252026.04 | 82.7 | |
| Gemini-2.5-Pro w/ GuarantRAGBackbone=Gemini-2.5-Pro, RAG Strategy=GuarantRAG, Retriever=BM252026.04 | 82.1 | |
| DeepSeek-V3 w/ P-RAGBackbone=DeepSeek-V3, RAG Strategy=P-RAG, Retriever=BM252026.04 | 81.6 | |
| DeepSeek-R1 w/ SelfRAGBackbone=DeepSeek-R1, RAG Strategy=SelfRAG, Retriever=BM252026.04 | 81 | |
| Claude-3.7-Sonnet w/ GuarantRAGBackbone=Claude-3.7-Sonnet, RAG Strategy=GuarantRAG, Retriever=BM252026.04 | 81 | |
| Gemini-2.5-Pro w/ P-RAGBackbone=Gemini-2.5-Pro, RAG Strategy=P-RAG, Retriever=BM252026.04 | 80.5 | |
| DeepSeek-V3 w/ SelfRAGBackbone=DeepSeek-V3, RAG Strategy=SelfRAG, Retriever=BM252026.04 | 80.1 | |
| DeepSeek-R1 w/ Standard RAGBackbone=DeepSeek-R1, RAG Strategy=Standard RAG, Retriever=BM252026.04 | 79.4 | |
| Claude-3.7-Sonnet w/ P-RAGBackbone=Claude-3.7-Sonnet, RAG Strategy=P-RAG, Retriever=BM252026.04 | 79.4 | |
| Gemini-2.5-Pro w/ SelfRAGBackbone=Gemini-2.5-Pro, RAG Strategy=SelfRAG, Retriever=BM252026.04 | 78.9 | |
| DeepSeek-V3 w/ Standard RAGBackbone=DeepSeek-V3, RAG Strategy=Standard RAG, Retriever=BM252026.04 | 78.3 | |
| Claude-3.7-Sonnet w/ SelfRAGBackbone=Claude-3.7-Sonnet, RAG Strategy=SelfRAG, Retriever=BM252026.04 | 77.8 | |
| Gemini-2.5-Pro w/ Standard RAGBackbone=Gemini-2.5-Pro, RAG Strategy=Standard RAG, Retriever=BM252026.04 | 77.2 | |
| DeepSeek-R1Backbone=DeepSeek-R1, RAG Strategy=Base, Retriever=BM252026.04 | 77.1 | |
| Claude-3.7-Sonnet w/ Standard RAGBackbone=Claude-3.7-Sonnet, RAG Strategy=Standard RAG, Retriever=BM252026.04 | 76.1 | |
| DeepSeek-V3Backbone=DeepSeek-V3, RAG Strategy=Base, Retriever=BM252026.04 | 75.8 | |
| Gemini-2.5-ProBackbone=Gemini-2.5-Pro, RAG Strategy=Base, Retriever=BM252026.04 | 74.9 | |
| Claude-3.7-SonnetBackbone=Claude-3.7-Sonnet, RAG Strategy=Base, Retriever=BM252026.04 | 73.6 |