List your company
ResearchFormerly WizWand

Closed Book Question Answering on Natural Questions (accuracy)

40.1AccuracyU-PaLM
8.91725.133.2Oct 20, 2022
Updated 8mo ago

Evaluation results

MethodLinks
U-PaLMSize=540B, FLOPS (ZFLOPS)=2529.7, Shot=few-shot2022.1040.1
PaLMSize=540B, FLOPS (ZFLOPS)=2527.2, Shot=few-shot2022.1036
ChinchillaSize=70B, FLOPS (ZFLOPS)=588, Shot=few-shot2022.1031.5
U-PaLMSize=62B, FLOPS (ZFLOPS)=298.7, Shot=few-shot2022.1030.5
PaLMSize=62B, FLOPS (ZFLOPS)=295.7, Shot=few-shot2022.1027.6
GopherSize=280B, FLOPS (ZFLOPS)=504, Shot=few-shot2022.1024.5
U-PaLMSize=540B, FLOPS (ZFLOPS)=2529.7, Shot=0-shot2022.1021.7
PaLMSize=540B, FLOPS (ZFLOPS)=2527.2, Shot=0-shot2022.1021.2
U-PaLMSize=62B, FLOPS (ZFLOPS)=298.7, Shot=0-shot2022.1018.7
PaLMSize=62B, FLOPS (ZFLOPS)=295.7, Shot=0-shot2022.1018.1
ChinchillaSize=70B, FLOPS (ZFLOPS)=588, Shot=0-shot2022.1016.6
GopherSize=280B, FLOPS (ZFLOPS)=504, Shot=0-shot2022.1010.1