Knowledge-intensive language tasks evaluation on Knowledge Intensive Tasks (test)
43.9NQRA-DIT 65B
Evaluation Results
| Method | Links | ||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| RA-DIT 65Binference_mode=5-shot in-context2023.10 | 43.9 | 64.9 | 75.1 | 23.2 | 40.7 | 90.7 | 55.8 | 72.4 | 68.4 | 17.3 | 51.8 | 55.2 | |
| RA-DIT 65Binference_mode=64-shot fine-tuned, Training strategy=multi-task fine-tuning2023.10 | 43.5 | — | 72.8 | — | 36.6 | 86.9 | 80.5 | 78.1 | 72.8 | 15.7 | — | 60.9 | |
| ATLASinference_mode=64-shot fine-tuned, Training strategy=task-specific models2023.10 | 42.4 | — | 74.5 | — | 34.7 | 87.1 | 66.5 | 74.9 | 58.9 | 15.5 | — | 56.8 | |
| LLAMA 65B REPLUGinference_mode=5-shot in-context2023.10 | 42.3 | 64.4 | 74.9 | 22.8 | 41.1 | 89.4 | 46.4 | 60.4 | 68.9 | 16.8 | 51.1 | 52.7 | |
| RA-DIT 65Binference_mode=0-shot2023.10 | 35.2 | 64.6 | 75.4 | 21.2 | 39.7 | 80.7 | 45.1 | 73.7 | 53.1 | 16.4 | 49.1 | 50.5 | |
| LLAMA 65Binference_mode=5-shot in-context2023.10 | 31.6 | 63.4 | 71.8 | 22.1 | 22.6 | 81.5 | 48.2 | 39.4 | 52.1 | 17.4 | 47.2 | 45 | |
| LLAMA 65B REPLUGinference_mode=0-shot2023.10 | 28.8 | 59.7 | 72.6 | 19.1 | 32 | 73.3 | 11.8 | 50.8 | 36.3 | 16.1 | 45.1 | 43.1 | |
| LLAMA 65Binference_mode=0-shot2023.10 | 5.2 | 51.2 | 55.8 | 19.5 | 12.5 | 59.3 | 0.6 | 6.7 | 1.3 | 15.6 | 32.9 | 22.8 |