Compression Capacity on PG-19 (test)
1,568Max TokensLlama-3.1-8B
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| Llama-3.1-8BNumber of trainable vectors=1, Backbone family=Llama2025.02 | 1,568 | 1,094.1 | 4,865.7 | |
| Llama-3.2-3BNumber of trainable vectors=1, Backbone family=Llama2025.02 | 1,024 | 720.3 | 3,292.2 | |
| Llama-3.2-1BNumber of trainable vectors=1, Backbone family=Llama2025.02 | 512 | 426.2 | 2,119.9 | |
| Pythia-1.4BNumber of trainable vectors=1, Backbone family=Pythia2025.02 | 160 | 158 | 792.8 | |
| Pythia-410MNumber of trainable vectors=1, Backbone family=Pythia2025.02 | 96 | 81.3 | 431.4 | |
| Pythia-160MNumber of trainable vectors=1, Backbone family=Pythia2025.02 | 80 | 70.9 | 396.4 |