Open-domain Question Answering on TriviaQA (Acc, F1, Tok)
77AccuracyChatGPT
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| ChatGPTGeneration Type=Direct Generation2024.12 | 77 | 52.9 | — | |
| RetroLLM (Ours)Generation Type=Retrieval within Generation, Backbone=Mistral-7B-Instruct2024.12 | 74.3 | 72.8 | 287 | |
| Iter-RetGenGeneration Type=Retrieval-augmented Generation, Retriever=E5-base-en2024.12 | 71 | 69.9 | 2,461 | |
| Naive RAGGeneration Type=Retrieval-augmented Generation, Retriever=E5-base-en2024.12 | 69.3 | 65.9 | 915 | |
| IRCoTGeneration Type=Retrieval-augmented Generation, Retriever=E5-base-en2024.12 | 66 | 66.1 | 1,715 | |
| REPLUGGeneration Type=Retrieval-augmented Generation, Retriever=E5-base-en2024.12 | 65.4 | 66.5 | 939 | |
| Adaptive-RAGGeneration Type=Retrieval-augmented Generation, Retriever=E5-base-en2024.12 | 65.1 | 65.6 | 958 | |
| Self-RAGGeneration Type=Retrieval-augmented Generation, Retriever=E5-base-en2024.12 | 64.1 | 53.4 | 1,267 | |
| Mistral-7BGeneration Type=Direct Generation2024.12 | 58.8 | 58.6 | 57 | |
| Llama3-8BGeneration Type=Direct Generation2024.12 | 56.1 | 60.2 | 52 | |
| Qwen2.5-7BGeneration Type=Direct Generation2024.12 | 45.1 | 48.1 | 54 |