List your company
ResearchFormerly WizWand

Reading Comprehension on Lambada

80.5AccuracyU-PaLM
5.72425.13744.5563.963Mar 29, 2022Sep 19, 2022Mar 12, 2023Sep 2, 2023Feb 23, 2024Aug 15, 2024Feb 5, 2025
Updated 8mo ago

Evaluation results

MethodLinks
U-PaLMSize=540B, FLOPS (ZFLOPS)=2529.7, Shot=0-shot2022.1080.5
U-PaLMSize=62B, FLOPS (ZFLOPS)=298.7, Shot=0-shot2022.1079.7
PaLMSize=540B, FLOPS (ZFLOPS)=2527.2, Shot=0-shot2022.1077.9
Chinchillaevaluation_mode=Zero-Shot2022.0377.4
ChinchillaSize=70B, FLOPS (ZFLOPS)=588, Shot=0-shot2022.1077.2
MT-NLG 530Bevaluation_mode=Zero-Shot2022.0376.6
GPT-3evaluation_mode=Zero-Shot2022.0376.2
PaLMSize=62B, FLOPS (ZFLOPS)=295.7, Shot=0-shot2022.1075.4
GopherSize=280B, FLOPS (ZFLOPS)=504, Shot=0-shot2022.1074.5
Gopherevaluation_mode=Zero-Shot2022.0374.5
LLaMA2Model=LLaMA2, Parameters=7B2025.0268.8
DiffuGPT-MModel=DiffuGPT-M, Parameters=355M2025.0260.5
DiffuLLaMA + P2-selfModel=DiffuLLaMA, Parameters=7B, Sampling Strategy=P2-self2025.0254.8
DiffuLLaMAModel=DiffuLLaMA, Parameters=7B, Sampling Strategy=ancestral sampling2025.0253.7
MDM + P2-selfModel=MDM, Parameters=1.1B, Sampling Strategy=P2-self2025.0252.9
MDMModel=MDM, Parameters=1.1B, Sampling Strategy=ancestral sampling2025.0252.7
DiffuGPT-SModel=DiffuGPT-S, Parameters=127M2025.0245
GPT-2Model=GPT-2, Parameters=1.5B2025.0244.6
TinyLlamaModel=TinyLlama, Parameters=1.1B2025.0243.2
GPT2-MModel=GPT2-M, Parameters=355M2025.0237.7
GPT2-SModel=GPT2-S, Parameters=127M2025.0225.9
SEDD-MModel=SEDD-M, Parameters=424M2025.0223.1
SEDD-SModel=SEDD-S, Parameters=170M2025.0212.4
Plaid1BModel=Plaid1B, Parameters=1.3B2025.028.6