Language Modeling on WikiText-2K
12.3PerplexityLLaMA2-7B
Evaluation results
| Method | Links | |
|---|---|---|
| LLaMA2-7BNumber of parameters=7B2024.07 | 12.3 | |
| TransAct-2.6BNumber of parameters=2.6B2024.07 | 13.8 | |
| Sheared-LLaMA-2.7BNumber of parameters=2.7B2024.07 | 14.7 | |
| LLM-Pruner-2.6BNumber of parameters=2.6B, Model Architecture=TransAct2024.07 | 15.1 | |
| TransAct-1.3BNumber of parameters=1.3B2024.07 | 17.3 | |
| Sheared-LLaMA-1.3BNumber of parameters=1.3B2024.07 | 18.1 | |
| LLM-Pruner-1.3BNumber of parameters=1.3B, Model Architecture=TransAct2024.07 | 20.5 |