Open-domain Question Answering on 2WIKI
48.9AccuracyRetroLLM (Ours)
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| RetroLLM (Ours)Generation Type=Retrieval within Generation, Backbone=Mistral-7B-Instruct2024.12 | 48.9 | 36.2 | 661 | |
| Naive RAGGeneration Type=Retrieval-augmented Generation, Retriever=E5-base-en2024.12 | 38.7 | 21.7 | 1,000 | |
| ChatGPTGeneration Type=Direct Generation2024.12 | 38 | 21.3 | — | |
| Mistral-7BGeneration Type=Direct Generation2024.12 | 36.5 | 18.7 | 58 | |
| Adaptive-RAGGeneration Type=Retrieval-augmented Generation, Retriever=E5-base-en2024.12 | 32.1 | 28.4 | 2,580 | |
| Self-RAGGeneration Type=Retrieval-augmented Generation, Retriever=E5-base-en2024.12 | 30.3 | 25.7 | 1,272 | |
| IRCoTGeneration Type=Retrieval-augmented Generation, Retriever=E5-base-en2024.12 | 29.4 | 32.4 | 1,707 | |
| Iter-RetGenGeneration Type=Retrieval-augmented Generation, Retriever=E5-base-en2024.12 | 29.2 | 21.5 | 2,669 | |
| REPLUGGeneration Type=Retrieval-augmented Generation, Retriever=E5-base-en2024.12 | 24.5 | 20.8 | 1,007 | |
| Qwen2.5-7BGeneration Type=Direct Generation2024.12 | 22.4 | 28.1 | 53 | |
| Llama3-8BGeneration Type=Direct Generation2024.12 | 20.9 | 24.3 | 54 |