Citation-based Question Answering on ALCE-ASQA v1 (test)
41.7EM RecallCaLF (MistralOrca 7B)
Evaluation Results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| CaLF (MistralOrca 7B)Strategy=Fine-tuning2024.06 | 41.7 | 40.3 | 84 | 34.5 | 81.5 | |
| GPT-4Model Version=gpt-4-06132024.06 | 41.3 | — | 67.1 | — | 71.9 | |
| AGREEBackbone=PaLM 2 text-bison-0012024.06 | 40.9 | — | — | — | 75.1 | |
| ChatGPTModel Version=gpt-3.5-turbo-03012024.06 | 40.4 | — | 66.6 | — | 73.1 | |
| MistralOrca 7BStrategy=In-context2024.06 | 40.2 | 38.7 | 54.7 | 31.9 | 55.6 | |
| MistralOrca 7BStrategy=Few-shot FT2024.06 | 38.4 | 38.4 | 78.6 | 29.9 | 62.6 | |
| CaLF (Llama2-7B-chat)Strategy=Fine-tuning2024.06 | 37.7 | 37.8 | 86 | 29.3 | 70.4 | |
| CaLF (Mistral-Instr. 7B)Strategy=Fine-tuning2024.06 | 36.4 | 37.2 | 84.5 | 30 | 76.2 | |
| Mistral-Instr. 7BStrategy=Few-shot FT2024.06 | 36.1 | 38.1 | 87.7 | 29.4 | 66.7 | |
| Llama2-7B-chatStrategy=In-context2024.06 | 35 | 35.9 | 77.8 | 25.7 | 49.9 | |
| Mistral-Instr. 7BStrategy=In-context2024.06 | 34.4 | 36.7 | 85.5 | 27.8 | 22.3 | |
| Blueprint (BP)Backbone=T5-3B2024.06 | 33.8 | — | — | — | 77.8 | |
| Llama2-7B-chatStrategy=Few-shot FT2024.06 | 32 | 34.9 | 69.2 | 22.3 | 55 | |
| Self-RAG 7B2024.06 | 30 | 35.7 | 74.3 | — | 67.3 |