Language Modeling on WikiText-103 (Efficiency)
6.59GFLOPs/tokenLTE
Evaluation results
| Method | Links | |
|---|---|---|
| LTEBackbone=LLaMA-7B, Permitted Quality Drop=PPL <= 0.52024.02 | 6.59 | |
| Deja VuBackbone=LLaMA-7B, Permitted Quality Drop=PPL <= 0.52024.02 | 7.92 | |
| R-LLaMA+MoEBackbone=LLaMA-7B, Permitted Quality Drop=PPL <= 0.52024.02 | 11.1 | |
| FullBackbone=LLaMA-7B, Permitted Quality Drop=PPL <= 0.52024.02 | 12.06 |