Language Modeling on Shakespeare 1M params (val)
4.79NLLTensor Cache
Evaluation Results
| Method | Links | |
|---|---|---|
| Tensor Cachecontext length L=2562026.05 | 4.79 | |
| Infini-attentioncontext length L=2562026.05 | 4.8 | |
| Full KVcontext length L=2562026.05 | 4.91 | |
| Window KVcontext length L=2562026.05 | 4.91 | |
| StreamingLLMcontext length L=2562026.05 | 4.91 | |
| Tensor Cachecontext length L=10242026.05 | 4.96 | |
| Infini-attentioncontext length L=10242026.05 | 5.14 | |
| Full KVcontext length L=10242026.05 | 5.24 | |
| Tensor Cachecontext length L=163842026.05 | 5.32 | |
| StreamingLLMcontext length L=10242026.05 | 5.47 | |
| Window KVcontext length L=10242026.05 | 5.48 | |
| Infini-attentioncontext length L=163842026.05 | 5.51 | |
| Tensor Cachecontext length L=20482026.05 | 5.6 | |
| Tensor Cachecontext length L=40962026.05 | 5.66 | |
| Tensor Cachecontext length L=81922026.05 | 5.7 | |
| Full KVcontext length L=163842026.05 | 5.73 | |
| StreamingLLMcontext length L=163842026.05 | 5.79 | |
| Infini-attentioncontext length L=20482026.05 | 5.79 | |
| Window KVcontext length L=163842026.05 | 5.82 | |
| Infini-attentioncontext length L=40962026.05 | 5.87 | |
| Infini-attentioncontext length L=81922026.05 | 5.92 | |
| Tensor Cachecontext length L=5122026.05 | 5.98 | |
| Full KVcontext length L=20482026.05 | 6.02 | |
| Infini-attentioncontext length L=5122026.05 | 6.06 | |
| Full KVcontext length L=40962026.05 | 6.09 | |
| Full KVcontext length L=5122026.05 | 6.1 | |
| Full KVcontext length L=81922026.05 | 6.13 | |
| Window KVcontext length L=20482026.05 | 6.16 | |
| StreamingLLMcontext length L=20482026.05 | 6.16 | |
| StreamingLLMcontext length L=81922026.05 | 6.21 | |
| Window KVcontext length L=81922026.05 | 6.24 | |
| StreamingLLMcontext length L=40962026.05 | 6.24 | |
| Window KVcontext length L=5122026.05 | 6.25 | |
| StreamingLLMcontext length L=5122026.05 | 6.25 | |
| Window KVcontext length L=40962026.05 | 6.27 |