Likelihood Evaluation on FineWeb sample-10BT (train)
24.06LikelihoodAC-GPT
Evaluation Results
| Method | Links | |
|---|---|---|
| AC-GPTBackbone=GPT-2 base (124M), rmax=0.4, Future context conditioning=true2026.06 | 24.06 | |
| σ-temporalBackbone=GPT-2 base (124M), rmax=0.4, Future context conditioning=true2026.06 | 24.82 | |
| GPTBackbone=GPT-2 base (124M), rmax=0.4, Future context conditioning=false2026.06 | 27.48 | |
| AC-GPTBackbone=GPT-2 base (124M), rmax=0.4, Future context conditioning=false2026.06 | 28.6 | |
| σ-temporalBackbone=GPT-2 base (124M), rmax=0.4, Future context conditioning=false2026.06 | 28.94 | |
| σ-randomBackbone=GPT-2 base (124M), rmax=0.4, Future context conditioning=true2026.06 | 51.3 | |
| σ-randomBackbone=GPT-2 base (124M), rmax=0.4, Future context conditioning=false2026.06 | 602.08 |