Language Modeling (EM) on Lambada
89.7EM AccuracyPaLM 540B
Evaluation Results
| Method | Links | |
|---|---|---|
| PaLM 540BEvaluation Protocol=Few-shot, Number of Shots=82022.04 | 89.7 | |
| Megatron-Turing NLG 530BEvaluation Protocol=Few-shot, Number of Shots=152022.04 | 87.2 | |
| PaLM 540BEvaluation Protocol=1-shot2022.04 | 81.8 | |
| GLaM 62B/64EEvaluation Protocol=1-shot2022.04 | 80.9 | |
| PaLMModel Size (Billions of Parameters)=540, Num Tokens (Billions of Tokens)=780, Training FLOP Count (Zettaflops)=2527.2, Evaluation Protocol=0-shot2022.04 | 77.9 | |
| PaLM 540BEvaluation Protocol=0-shot2022.04 | 77.9 | |
| ChinchillaEvaluation Protocol=0-shot2022.04 | 77.7 | |
| ChinchillaModel Size (Billions of Parameters)=70, Num Tokens (Billions of Tokens)=1400, Training FLOP Count (Zettaflops)=588.0, Evaluation Protocol=0-shot2022.04 | 77.4 | |
| PaLMModel Size (Billions of Parameters)=62, Num Tokens (Billions of Tokens)=795, Training FLOP Count (Zettaflops)=295.7, Evaluation Protocol=0-shot2022.04 | 75.4 | |
| GopherModel Size (Billions of Parameters)=280, Num Tokens (Billions of Tokens)=300, Training FLOP Count (Zettaflops)=504.0, Evaluation Protocol=0-shot2022.04 | 74.5 | |
| PaLMModel Size (Billions of Parameters)=8, Num Tokens (Billions of Tokens)=780, Training FLOP Count (Zettaflops)=37.4, Evaluation Protocol=0-shot2022.04 | 69.5 | |
| DoGraphBackbone=GPT-2 Medium, Pre-training Dataset=SlimPajama2026.04 | 14.5 | |
| RegMixBackbone=GPT-2 Medium, Pre-training Dataset=SlimPajama2026.04 | 14 | |
| Dynamic Loss-BasedBackbone=GPT-2 Medium, Pre-training Dataset=SlimPajama2026.04 | 13.4 | |
| DoReMiBackbone=GPT-2 Medium, Pre-training Dataset=SlimPajama2026.04 | 12.7 | |
| Data Mixing LawBackbone=GPT-2 Medium, Pre-training Dataset=SlimPajama2026.04 | 12 | |
| UniformBackbone=GPT-2 Medium, Pre-training Dataset=SlimPajama2026.04 | 11.6 | |
| DOGEBackbone=GPT-2 Medium, Pre-training Dataset=SlimPajama2026.04 | 11.6 |