Closed-book Question Answering on Natural Questions (test)
29.9AccuracyGPT-3
Evaluation results
| Method | Links | |
|---|---|---|
| GPT-3shots=64-shot2021.12 | 29.9 | |
| Gopher 280Bshots=64-shot, parameters=280B2021.12 | 28.2 | |
| Gopher 280Bshots=5-shots, parameters=280B2021.12 | 24.5 | |
| GPT-3shots=1-shot2021.12 | 23 | |
| Gopher 280Bshots=1-shot, parameters=280B2021.12 | 20.1 | |
| Gopher 7.1Bshots=64-shot, parameters=7.1B2021.12 | 16.5 | |
| GPT-3shots=0-shot2021.12 | 14.6 | |
| Gopher 7.1Bshots=5-shots, parameters=7.1B2021.12 | 13.4 | |
| Gopher 7.1Bshots=1-shot, parameters=7.1B2021.12 | 10.8 | |
| Gopher 280Bshots=0-shot, parameters=280B2021.12 | 10.1 | |
| Gopher 1Bshots=64-shot, parameters=1B2021.12 | 8.1 | |
| Gopher 1Bshots=5-shots, parameters=1B2021.12 | 6.8 | |
| Gopher 7.1Bshots=0-shot, parameters=7.1B2021.12 | 6.1 | |
| Gopher 1Bshots=1-shot, parameters=1B2021.12 | 4.8 | |
| Gopher 417Mshots=64-shot, parameters=417M2021.12 | 3.4 | |
| Gopher 417Mshots=5-shots, parameters=417M2021.12 | 2.8 | |
| Gopher 1Bshots=0-shot, parameters=1B2021.12 | 2.4 | |
| Gopher 417Mshots=1-shot, parameters=417M2021.12 | 1.9 | |
| Gopher 117Mshots=64-shot, parameters=117M2021.12 | 1.8 | |
| Gopher 117Mshots=5-shots, parameters=117M2021.12 | 1.7 | |
| Gopher 117Mshots=1-shot, parameters=117M2021.12 | 0.9 | |
| Gopher 44Mshots=5-shots, parameters=44M2021.12 | 0.9 | |
| Gopher 44Mshots=64-shot, parameters=44M2021.12 | 0.8 | |
| Gopher 417Mshots=0-shot, parameters=417M2021.12 | 0.4 | |
| Gopher 117Mshots=0-shot, parameters=117M2021.12 | 0.1 | |
| Gopher 44Mshots=1-shot, parameters=44M2021.12 | 0.1 | |
| Gopher 44Mshots=0-shot, parameters=44M2021.12 | 0 |