List your company
ResearchFormerly WizWand

Sequence Modeling on Minipile (pretraining)

17.32PerplexityLayUp
16.945219.475122.00524.5349Oct 8, 2024
Updated 4mo ago

Evaluation results

MethodLinks
LayUpArchitecture=GPT-2 Medium, Evaluation Protocol=Pretraining2024.1017.3238,624.33
GoSGDArchitecture=GPT-2 Medium, Evaluation Protocol=Pretraining2024.1017.5241,147.67
SlowMoArchitecture=GPT-2 Medium, Evaluation Protocol=Pretraining2024.1017.7252,687
CO2Architecture=GPT-2 Medium, Evaluation Protocol=Pretraining2024.1017.7551,508
DDPArchitecture=GPT-2 Medium, Evaluation Protocol=Pretraining2024.1017.8540,658.05
AD-PSGDArchitecture=GPT-2 Medium, Evaluation Protocol=Pretraining2024.1026.6938,093.67