Language Modeling on Proof-Pile (PPL and LongPPL)
2.915PPLElastic Memory_exp
Evaluation Results
| Method | Links | ||
|---|---|---|---|
| Elastic Memory_expMem. Size=0.8M, Add. Params=02026.02 | 2.915 | 2.586 | |
| MelodiMem. Size=0.8M, Add. Params=1.5M2026.02 | 2.94 | 2.98 | |
| Infini-TransformerMem. Size=0.8M, Add. Params=962026.02 | 2.95 | 2.736 | |
| Elastic Memory_uniMem. Size=0.8M, Add. Params=02026.02 | 2.958 | 2.56 | |
| Mem. TransformerMem. Size=0.8M, Add. Params=962026.02 | 3.305 | 3.843 | |
| Transformer++Mem. Size=0, Add. Params=02026.02 | 3.322 | 3.866 |