Language Modeling on GPT-2 (val)
3.48Base CEGPT-2
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| GPT-22025.12 | 3.48 | — | — | |
| Sparse Attention Post-TrainingTarget Cross-Entropy (τ)=3.5, Base Model=GPT-22025.12 | — | 3.501 | 0.22 |
| Method | Links | |||
|---|---|---|---|---|
| GPT-22025.12 | 3.48 | — | — | |
| Sparse Attention Post-TrainingTarget Cross-Entropy (τ)=3.5, Base Model=GPT-22025.12 | — | 3.501 | 0.22 |