Long-form Question Answering on ASQA
91.18str-emCLARION
Evaluation results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| CLARIONbackbones=averaged over three LLMs2025.09 | 91.18 | — | — | — | 48.78 | |
| ReActbackbones=averaged over three LLMs2025.09 | 82.34 | — | — | — | 40.83 | |
| NaiveRAGbackbones=averaged over three LLMs2025.09 | 75.95 | — | — | — | 38.5 | |
| DIVAbackbones=averaged over three LLMs2025.09 | 71.7 | — | — | — | 32.92 | |
| AmberRetrieval Strategy=Amber, Backbone=GPT-3.52025.02 | 51.3 | 26.3 | — | — | — | |
| AmberRetrieval Strategy=Amber, Backbone=Qwen2-7b2025.02 | 49.7 | 25.2 | — | — | — | |
| VanillaRetrieval Strategy=Single-time RAG, Backbone=GPT-3.52025.02 | 47.77 | 21.62 | — | — | — | |
| AmberRetrieval Strategy=Amber, Backbone=Llama3-8b2025.02 | 44.7 | 18.8 | — | — | — | |
| Adaptive-NoteRetrieval Strategy=Adaptive RAG, Backbone=Llama3-8b2025.02 | 43.7 | 17.7 | — | — | — | |
| VanillaRetrieval Strategy=Single-time RAG, Backbone=Qwen2-7b2025.02 | 43.5 | 18.5 | — | — | — | |
| Self-RefineRetrieval Strategy=Single-time RAG, Backbone=Llama3-8b2025.02 | 42.1 | 16.6 | — | — | — | |
| Adaptive-RAGRetrieval Strategy=Adaptive RAG, Backbone=Llama3-8b2025.02 | 42.1 | 15.8 | — | — | — | |
| Self-RAGRetrieval Strategy=Adaptive RAG, Backbone=Llama3-8b2025.02 | 40.9 | 16.5 | — | — | — | |
| Chain-of-noteRetrieval Strategy=Single-time RAG, Backbone=Llama3-8b2025.02 | 40.3 | 15.6 | — | — | — | |
| VanillaRetrieval Strategy=Single-time RAG, Backbone=Llama3-8b2025.02 | 38.7 | 13.7 | — | — | — | |
| NORRetrieval Strategy=No Retrieval, Backbone=Llama3-8b2025.02 | 35.5 | 8.9 | — | — | — | |
| Self-RerankRetrieval Strategy=Single-time RAG, Backbone=Llama3-8b2025.02 | 35 | 11.4 | — | — | — | |
| FLARERetrieval Strategy=Adaptive RAG, Backbone=Llama3-8b2025.02 | 34.9 | 9.5 | — | — | — | |
| ReActRetrieval Strategy=Adaptive RAG, Backbone=Llama3-8b2025.02 | 32.9 | 8.3 | — | — | — | |
| BaselinePreference Category=Baseline answers from dataset2024.07 | — | — | 0 | 40 | — | |
| Error-Informed Refinement (EIR)Preference Category=Refined answers2024.07 | — | — | 18 | 60 | — | |
| TiePreference Category=Tie (no preference)2024.07 | — | — | 82 | 0 | — |