Language Modeling on WikiText-4K
11.7PerplexityLLaMA2-7B
Evaluation results
| Method | Links | |
|---|---|---|
| LLaMA2-7BNumber of parameters=7B2024.07 | 11.7 | |
| TransAct-2.6BNumber of parameters=2.6B2024.07 | 13.1 | |
| Sheared-LLaMA-2.7BNumber of parameters=2.7B2024.07 | 13.9 | |
| LLM-Pruner-2.6BNumber of parameters=2.6B, Model Architecture=TransAct2024.07 | 14.2 | |
| TransAct-1.3BNumber of parameters=1.3B2024.07 | 16.3 | |
| Sheared-LLaMA-1.3BNumber of parameters=1.3B2024.07 | 17.1 | |
| LLM-Pruner-1.3BNumber of parameters=1.3B, Model Architecture=TransAct2024.07 | 19.8 |