Reading Comprehension on Lambada
80.5AccuracyU-PaLM
Evaluation Results
| Method | Links | |
|---|---|---|
| U-PaLMSize=540B, FLOPS (ZFLOPS)=2529.7, Shot=0-shot2022.10 | 80.5 | |
| U-PaLMSize=62B, FLOPS (ZFLOPS)=298.7, Shot=0-shot2022.10 | 79.7 | |
| PaLMSize=540B, FLOPS (ZFLOPS)=2527.2, Shot=0-shot2022.10 | 77.9 | |
| Chinchillaevaluation_mode=Zero-Shot2022.03 | 77.4 | |
| ChinchillaSize=70B, FLOPS (ZFLOPS)=588, Shot=0-shot2022.10 | 77.2 | |
| MT-NLG 530Bevaluation_mode=Zero-Shot2022.03 | 76.6 | |
| GPT-3evaluation_mode=Zero-Shot2022.03 | 76.2 | |
| PaLMSize=62B, FLOPS (ZFLOPS)=295.7, Shot=0-shot2022.10 | 75.4 | |
| GopherSize=280B, FLOPS (ZFLOPS)=504, Shot=0-shot2022.10 | 74.5 | |
| Gopherevaluation_mode=Zero-Shot2022.03 | 74.5 | |
| LLaMA2Model=LLaMA2, Parameters=7B2025.02 | 68.8 | |
| DiffuGPT-MModel=DiffuGPT-M, Parameters=355M2025.02 | 60.5 | |
| DiffuLLaMA + P2-selfModel=DiffuLLaMA, Parameters=7B, Sampling Strategy=P2-self2025.02 | 54.8 | |
| DiffuLLaMAModel=DiffuLLaMA, Parameters=7B, Sampling Strategy=ancestral sampling2025.02 | 53.7 | |
| MDM + P2-selfModel=MDM, Parameters=1.1B, Sampling Strategy=P2-self2025.02 | 52.9 | |
| MDMModel=MDM, Parameters=1.1B, Sampling Strategy=ancestral sampling2025.02 | 52.7 | |
| DiffuGPT-SModel=DiffuGPT-S, Parameters=127M2025.02 | 45 | |
| GPT-2Model=GPT-2, Parameters=1.5B2025.02 | 44.6 | |
| TinyLlamaModel=TinyLlama, Parameters=1.1B2025.02 | 43.2 | |
| GPT2-MModel=GPT2-M, Parameters=355M2025.02 | 37.7 | |
| GPT2-SModel=GPT2-S, Parameters=127M2025.02 | 25.9 | |
| SEDD-MModel=SEDD-M, Parameters=424M2025.02 | 23.1 | |
| SEDD-SModel=SEDD-S, Parameters=170M2025.02 | 12.4 | |
| Plaid1BModel=Plaid1B, Parameters=1.3B2025.02 | 8.6 |