Infilling Likelihood Evaluation on FineWeb sample-10BT
805.77Likelihoodσ-random
Evaluation Results
| Method | Links | |
|---|---|---|
| σ-randomBackbone=GPT-2 base (124M), rmax=0.4, Future context conditioning=true2026.06 | 805.77 | |
| σ-randomBackbone=GPT-2 base (124M), rmax=0.4, Future context conditioning=false2026.06 | 650.26 | |
| σ-temporalBackbone=GPT-2 base (124M), rmax=0.4, Future context conditioning=false2026.06 | 28.08 | |
| AC-GPTBackbone=GPT-2 base (124M), rmax=0.4, Future context conditioning=false2026.06 | 27.7 | |
| σ-temporalBackbone=GPT-2 base (124M), rmax=0.4, Future context conditioning=true2026.06 | 26.94 | |
| GPTBackbone=GPT-2 base (124M), rmax=0.4, Future context conditioning=false2026.06 | 26.85 | |
| AC-GPTBackbone=GPT-2 base (124M), rmax=0.4, Future context conditioning=true2026.06 | 26.74 |