Language Modeling on SlimPajama 10M (dev)
9.219PerplexityTinyLlama
Evaluation results
| Method | Links | |
|---|---|---|
| TinyLlama2024.05 | 9.219 | |
| Layer-Condensed KV Cachew=102024.05 | 9.265 | |
| Layer-Condensed KV Cachew=22024.05 | 9.746 |
| Method | Links | |
|---|---|---|
| TinyLlama2024.05 | 9.219 | |
| Layer-Condensed KV Cachew=102024.05 | 9.265 | |
| Layer-Condensed KV Cachew=22024.05 | 9.746 |