Open-domain Question Answering on PopQA (Acc, F1, Tok)
65.7AccuracyRetroLLM (Ours)
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| RetroLLM (Ours)Generation Type=Retrieval within Generation, Backbone=Mistral-7B-Instruct2024.12 | 65.7 | 43 | 355 | |
| IRCoTGeneration Type=Retrieval-augmented Generation, Retriever=E5-base-en2024.12 | 59.8 | 45.6 | 1,667 | |
| Adaptive-RAGGeneration Type=Retrieval-augmented Generation, Retriever=E5-base-en2024.12 | 58.3 | 40.4 | 1,681 | |
| Iter-RetGenGeneration Type=Retrieval-augmented Generation, Retriever=E5-base-en2024.12 | 51.7 | 47.5 | 2,509 | |
| Naive RAGGeneration Type=Retrieval-augmented Generation, Retriever=E5-base-en2024.12 | 47.7 | 38.6 | 944 | |
| Self-RAGGeneration Type=Retrieval-augmented Generation, Retriever=E5-base-en2024.12 | 39.7 | 32.7 | 1,236 | |
| REPLUGGeneration Type=Retrieval-augmented Generation, Retriever=E5-base-en2024.12 | 38.2 | 37 | 921 | |
| ChatGPTGeneration Type=Direct Generation2024.12 | 26.6 | 13.2 | — | |
| Mistral-7BGeneration Type=Direct Generation2024.12 | 25.8 | 25.2 | 45 | |
| Llama3-8BGeneration Type=Direct Generation2024.12 | 24.2 | 26.4 | 43 | |
| Qwen2.5-7BGeneration Type=Direct Generation2024.12 | 17.1 | 18.7 | 45 |