List your company
ResearchFormerly WizWand

Language Modeling on PG19 bytes (test)

0.935Bits Per TokenFeedback:fixed:skip
0.9320.952250.97250.99275Mar 11, 2022
Updated 8mo ago

Evaluation results

MethodLinks
Feedback:fixed:skipsegment length=4096, window length=512, step time (relative)=1.352022.030.935
Memorizing Trans. 64ksegment length=512, window length=512, step time (relative)=1.942022.030.95
Rec:fixed:skipsegment length=4096, window length=512, step time (relative)=0.992022.030.952
Rec:fixed:dualsegment length=4096, window length=512, step time (relative)=1.012022.030.957
Rec:lstm:singlesegment length=4096, window length=512, step time (relative)=1.052022.030.962
Rec:fixed:singlesegment length=4096, window length=512, step time (relative)=1.022022.030.966
Rec:lstm:skipsegment length=4096, window length=512, step time (relative)=12022.030.969
Feedback:lstm:singlesegment length=4096, window length=512, step time (relative)=1.42022.030.977
Rec:lstm:dualsegment length=4096, window length=512, step time (relative)=1.062022.030.985
Slide:12Lsegment length=4096, window length=512, step time (relative)=0.932022.030.989
Slide:13Lsegment length=4096, window length=512, step time (relative)=12022.030.989
XL:2048segment length=2048, window length=2048, step time (relative)=2.112022.030.99
XL:1024segment length=1024, window length=1024, step time (relative)=1.22022.030.997
XL:512segment length=512, window length=512, step time (relative)=0.882022.031.01