Open-domain Question Answering on Natural Questions (Metric)
39.6QA Metric ScorePaLM
Evaluation results
| Method | Links | |
|---|---|---|
| PaLMModel Size (Billions of Parameters)=540, Num Tokens (Billions of Tokens)=780, Training FLOP Count (Zettaflops)=2527.2, Evaluation Protocol=few-shot2022.04 | 39.6 | |
| ChinchillaModel Size (Billions of Parameters)=70, Num Tokens (Billions of Tokens)=1400, Training FLOP Count (Zettaflops)=588.0, Evaluation Protocol=few-shot2022.04 | 31.5 | |
| PaLMModel Size (Billions of Parameters)=62, Num Tokens (Billions of Tokens)=795, Training FLOP Count (Zettaflops)=295.7, Evaluation Protocol=few-shot2022.04 | 27.6 | |
| GopherModel Size (Billions of Parameters)=280, Num Tokens (Billions of Tokens)=300, Training FLOP Count (Zettaflops)=504.0, Evaluation Protocol=few-shot2022.04 | 24.5 | |
| PaLMModel Size (Billions of Parameters)=540, Num Tokens (Billions of Tokens)=780, Training FLOP Count (Zettaflops)=2527.2, Evaluation Protocol=0-shot2022.04 | 21.2 | |
| PaLMModel Size (Billions of Parameters)=62, Num Tokens (Billions of Tokens)=795, Training FLOP Count (Zettaflops)=295.7, Evaluation Protocol=0-shot2022.04 | 18.1 | |
| ChinchillaModel Size (Billions of Parameters)=70, Num Tokens (Billions of Tokens)=1400, Training FLOP Count (Zettaflops)=588.0, Evaluation Protocol=0-shot2022.04 | 16.6 | |
| PaLMModel Size (Billions of Parameters)=8, Num Tokens (Billions of Tokens)=780, Training FLOP Count (Zettaflops)=37.4, Evaluation Protocol=few-shot2022.04 | 14.6 | |
| GopherModel Size (Billions of Parameters)=280, Num Tokens (Billions of Tokens)=300, Training FLOP Count (Zettaflops)=504.0, Evaluation Protocol=0-shot2022.04 | 10.1 | |
| PaLMModel Size (Billions of Parameters)=8, Num Tokens (Billions of Tokens)=780, Training FLOP Count (Zettaflops)=37.4, Evaluation Protocol=0-shot2022.04 | 8.4 |