Word Prediction on CBT
53.67AccuracyPoPE
Evaluation results
| Method | Links | |
|---|---|---|
| PoPEModel size=774M, Zero-shot=true, Pre-training dataset=OpenWebText2025.09 | 53.67 | |
| RoPEModel size=774M, Zero-shot=true, Pre-training dataset=OpenWebText2025.09 | 52.56 | |
| PoPEModel size=253M, Zero-shot=true, Pre-training dataset=OpenWebText2025.09 | 49.39 | |
| RoPEModel size=253M, Zero-shot=true, Pre-training dataset=OpenWebText2025.09 | 48.51 | |
| PoPEModel size=124M, Zero-shot=true, Pre-training dataset=OpenWebText2025.09 | 44.43 | |
| RoPEModel size=124M, Zero-shot=true, Pre-training dataset=OpenWebText2025.09 | 38.8 |