Citation-based Question Answering on ALCE-ELI5 v1 (test)
20.8EM Rec.MistralOrca 7B
Evaluation results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| MistralOrca 7BStrategy=In-context2024.06 | 20.8 | 20.9 | 29.3 | 12.5 | 43.3 | |
| CaLF (Mistral-Instr. 7B)Strategy=Fine-tuning2024.06 | 19.5 | 21.8 | 58.2 | 13.1 | 66 | |
| Mistral-Instr. 7BStrategy=In-context2024.06 | 19.1 | 21.6 | 43.8 | 11.1 | 19.5 | |
| CaLF (MistralOrca 7B)Strategy=Fine-tuning2024.06 | 18.4 | 20.4 | 62.7 | 13.1 | 73.1 | |
| Llama2-7B-chatStrategy=Few-shot FT2024.06 | 17.8 | 21.3 | 58.2 | 11.2 | 48.7 | |
| Llama2-7B-chatStrategy=In-context2024.06 | 17.7 | 20.5 | 36.2 | 10.8 | 38.2 | |
| MistralOrca 7BStrategy=Few-shot FT2024.06 | 17.3 | 19.4 | 60.5 | 10.9 | 57.7 | |
| CaLF (Llama2-7B-chat)Strategy=Fine-tuning2024.06 | 17 | 20.8 | 59.6 | 11.9 | 66.5 | |
| Mistral-Instr. 7BStrategy=Few-shot FT2024.06 | 15.5 | 20.5 | 48 | 10 | 49.9 | |
| GPT-4Model Version=gpt-4-06132024.06 | 14.2 | — | 38.4 | — | 46.9 | |
| ChatGPTModel Version=gpt-3.5-turbo-03012024.06 | 12 | — | 57.2 | — | 50.5 | |
| Self-RAG 7B2024.06 | 9.7 | 16.9 | 32.6 | 5.4 | 27.6 | |
| Blueprint (BP)Backbone=T5-3B2024.06 | 5.2 | — | — | — | 60.9 |