Language Modeling on PG19 bytes (test)
0.935Bits Per TokenFeedback:fixed:skip
Evaluation Results
| Method | Links | |
|---|---|---|
| Feedback:fixed:skipsegment length=4096, window length=512, step time (relative)=1.352022.03 | 0.935 | |
| Memorizing Trans. 64ksegment length=512, window length=512, step time (relative)=1.942022.03 | 0.95 | |
| Rec:fixed:skipsegment length=4096, window length=512, step time (relative)=0.992022.03 | 0.952 | |
| Rec:fixed:dualsegment length=4096, window length=512, step time (relative)=1.012022.03 | 0.957 | |
| Rec:lstm:singlesegment length=4096, window length=512, step time (relative)=1.052022.03 | 0.962 | |
| Rec:fixed:singlesegment length=4096, window length=512, step time (relative)=1.022022.03 | 0.966 | |
| Rec:lstm:skipsegment length=4096, window length=512, step time (relative)=12022.03 | 0.969 | |
| Feedback:lstm:singlesegment length=4096, window length=512, step time (relative)=1.42022.03 | 0.977 | |
| Rec:lstm:dualsegment length=4096, window length=512, step time (relative)=1.062022.03 | 0.985 | |
| Slide:12Lsegment length=4096, window length=512, step time (relative)=0.932022.03 | 0.989 | |
| Slide:13Lsegment length=4096, window length=512, step time (relative)=12022.03 | 0.989 | |
| XL:2048segment length=2048, window length=2048, step time (relative)=2.112022.03 | 0.99 | |
| XL:1024segment length=1024, window length=1024, step time (relative)=1.22022.03 | 0.997 | |
| XL:512segment length=512, window length=512, step time (relative)=0.882022.03 | 1.01 |