Natural Language Understanding on NLU 21 tasks one-shot
61.7Average ScoreLion
Evaluation results
| Method | Links | |
|---|---|---|
| LionModel Size=7.5B, Optimizer=Lion, #Tokens=300B, Evaluation Protocol=One-shot2023.02 | 61.7 | |
| AdafactorModel Size=7.5B, Optimizer=Adafactor, #Tokens=300B, Evaluation Protocol=One-shot2023.02 | 61.3 | |
| PaLMModel Size=8B, Optimizer=not specified, #Tokens=780B, Evaluation Protocol=One-shot2023.02 | 59.4 | |
| GPT-3Model Size=6.7B, Optimizer=AdamW (implied), #Tokens=300B, Evaluation Protocol=One-shot2023.02 | 58.5 | |
| LionModel Size=2.1B, Optimizer=Lion, #Tokens=300B, Evaluation Protocol=One-shot2023.02 | 57.4 | |
| AdafactorModel Size=2.1B, Optimizer=Adafactor, #Tokens=300B, Evaluation Protocol=One-shot2023.02 | 56.8 | |
| LionModel Size=1.1B, Optimizer=Lion, #Tokens=300B, Evaluation Protocol=One-shot2023.02 | 53.9 | |
| AdafactorModel Size=1.1B, Optimizer=Adafactor, #Tokens=300B, Evaluation Protocol=One-shot2023.02 | 53.2 |