Closed Book Question Answering on Natural Questions (accuracy)
40.1AccuracyU-PaLM
Evaluation Results
| Method | Links | |
|---|---|---|
| U-PaLMSize=540B, FLOPS (ZFLOPS)=2529.7, Shot=few-shot2022.10 | 40.1 | |
| PaLMSize=540B, FLOPS (ZFLOPS)=2527.2, Shot=few-shot2022.10 | 36 | |
| ChinchillaSize=70B, FLOPS (ZFLOPS)=588, Shot=few-shot2022.10 | 31.5 | |
| U-PaLMSize=62B, FLOPS (ZFLOPS)=298.7, Shot=few-shot2022.10 | 30.5 | |
| PaLMSize=62B, FLOPS (ZFLOPS)=295.7, Shot=few-shot2022.10 | 27.6 | |
| GopherSize=280B, FLOPS (ZFLOPS)=504, Shot=few-shot2022.10 | 24.5 | |
| U-PaLMSize=540B, FLOPS (ZFLOPS)=2529.7, Shot=0-shot2022.10 | 21.7 | |
| PaLMSize=540B, FLOPS (ZFLOPS)=2527.2, Shot=0-shot2022.10 | 21.2 | |
| U-PaLMSize=62B, FLOPS (ZFLOPS)=298.7, Shot=0-shot2022.10 | 18.7 | |
| PaLMSize=62B, FLOPS (ZFLOPS)=295.7, Shot=0-shot2022.10 | 18.1 | |
| ChinchillaSize=70B, FLOPS (ZFLOPS)=588, Shot=0-shot2022.10 | 16.6 | |
| GopherSize=280B, FLOPS (ZFLOPS)=504, Shot=0-shot2022.10 | 10.1 |