Language Modeling on SlimPajama 10M (dev)
9.219PerplexityTinyLlama
Evaluation Results
| Method | Links | |
|---|---|---|
| TinyLlama2024.05 | 9.219 | |
| Layer-Condensed KV Cachew=102024.05 | 9.265 | |
| Layer-Condensed KV Cachew=22024.05 | 9.746 |
Loading the SOTA2 catalog…
| Method | Links | |
|---|---|---|
| TinyLlama2024.05 | 9.219 | |
| Layer-Condensed KV Cachew=102024.05 | 9.265 | |
| Layer-Condensed KV Cachew=22024.05 | 9.746 |