Next token prediction on OpenWebText
7.61BPDGPT2 (124M)
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| GPT2 (124M)Model size=124M (small), Backbone=GPT2, Quantization=Post-training quantization [31]2024.07 | 7.61 | 0.7482 | 0.2698 | |
| GPT2 (355M)Model size=354M (medium), Backbone=GPT2, Quantization=Post-training quantization [31]2024.07 | 8.5 | 0.7919 | 0.3272 | |
| GPT2 (774M)Model size=773M (large), Backbone=GPT2, Quantization=Post-training quantization [31]2024.07 | 10.47 | 0.895 | 0.4423 | |
| Random Guess2024.07 | 15.62 | 0.9999 | 0.998 |