Question Answering on KILT Benchmark Natural Questions, TriviaQA, HotpotQA (test)
85.6EM Score (TriviaQA)Llama3-ChatQA-1.5-70B
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| Llama3-ChatQA-1.5-70BZero-shot=true, Retriever=Dragon, Parameters=70B, Contexts=top-15/20/25 (best)2024.01 | 85.6 | 58.7 | 47 | 42.2 | |
| Llama3-Instruct-70BZero-shot=true, Retriever=Dragon, Parameters=70B, Contexts=top-15/20/25 (best)2024.01 | 82.4 | 53.6 | 42.7 | 35.6 | |
| Llama3-ChatQA-1.5-8BZero-shot=true, Retriever=Dragon, Parameters=8B, Contexts=top-15/20/25 (best)2024.01 | 81 | 52.3 | 42.4 | 33.5 | |
| RA-DIT (65B)Zero-shot=true, Parameters=65B2024.01 | 75.4 | 50.1 | 35.2 | 39.7 | |
| RePlug (65B)Zero-shot=true, Parameters=65B2024.01 | 72.6 | 44.5 | 28.8 | 32 | |
| Llama3-Instruct-8BZero-shot=true, Retriever=Dragon, Parameters=8B, Contexts=top-15/20/25 (best)2024.01 | 70.7 | 42.5 | 30.9 | 26 | |
| Raven (11B)Zero-shot=true, Parameters=11B2024.01 | 65.7 | — | 29.6 | — | |
| RECOMP (20B)Zero-shot=true, Parameters=20B2024.01 | 59 | 42.1 | 37 | 30.4 | |
| Atlas (11B)Zero-shot=true, Parameters=11B2024.01 | 56.9 | 39.4 | 26.7 | 34.7 | |
| InstructRetro (43B)Zero-shot=true, Parameters=43B2024.01 | 38.9 | — | — | 65.6 |