Language Modeling on PG19 tokens (test)
3.49Bits per TokenFeedback:fixed:skip
Evaluation Results
| Method | Links | |
|---|---|---|
| Feedback:fixed:skipsegment length=4096, window length=512, step time (relative)=1.352022.03 | 3.49 | |
| Feedback:lstm:singlesegment length=4096, window length=512, step time (relative)=1.42022.03 | 3.5 | |
| Rec:fixed:dualsegment length=4096, window length=512, step time (relative)=1.012022.03 | 3.52 | |
| Rec:fixed:skipsegment length=4096, window length=512, step time (relative)=0.992022.03 | 3.53 | |
| Memorizing Trans. 64ksegment length=512, window length=512, step time (relative)=1.942022.03 | 3.53 | |
| Rec:lstm:dualsegment length=4096, window length=512, step time (relative)=1.062022.03 | 3.54 | |
| Rec:lstm:singlesegment length=4096, window length=512, step time (relative)=1.052022.03 | 3.54 | |
| Rec:lstm:skipsegment length=4096, window length=512, step time (relative)=12022.03 | 3.56 | |
| XL:2048segment length=2048, window length=2048, step time (relative)=2.112022.03 | 3.58 | |
| Slide:13Lsegment length=4096, window length=512, step time (relative)=12022.03 | 3.58 | |
| Rec:fixed:singlesegment length=4096, window length=512, step time (relative)=1.022022.03 | 3.58 | |
| XL:1024segment length=1024, window length=1024, step time (relative)=1.22022.03 | 3.59 | |
| Slide:12Lsegment length=4096, window length=512, step time (relative)=0.932022.03 | 3.6 | |
| XL:512segment length=512, window length=512, step time (relative)=0.882022.03 | 3.62 |