Language Modeling on WikiText 1K
13.8PerplexityLLaMA2-7B
Evaluation results
| Method | Links | |
|---|---|---|
| LLaMA2-7BNumber of parameters=7B2024.07 | 13.8 | |
| TransAct-2.6BNumber of parameters=2.6B2024.07 | 15.3 | |
| LLM-Pruner-2.6BNumber of parameters=2.6B, Model Architecture=TransAct2024.07 | 16 | |
| Sheared-LLaMA-2.7BNumber of parameters=2.7B2024.07 | 16.3 | |
| TransAct-1.3BNumber of parameters=1.3B2024.07 | 19.3 | |
| Sheared-LLaMA-1.3BNumber of parameters=1.3B2024.07 | 20 | |
| LLM-Pruner-1.3BNumber of parameters=1.3B, Model Architecture=TransAct2024.07 | 21.7 |