Compression Capacity on Random GloVe (test)
792Max TokensLlama-3.1-8B
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| Llama-3.1-8BNumber of trainable vectors=1, Backbone family=Llama2025.02 | 792 | 623.2 | 4,541.2 | |
| Llama-3.2-3BNumber of trainable vectors=1, Backbone family=Llama2025.02 | 460 | 456.9 | 3,382.6 | |
| Llama-3.2-1BNumber of trainable vectors=1, Backbone family=Llama2025.02 | 316 | 294.9 | 2,286.2 | |
| Pythia-1.4BNumber of trainable vectors=1, Backbone family=Pythia2025.02 | 139 | 144.4 | 1,108.2 | |
| Pythia-410MNumber of trainable vectors=1, Backbone family=Pythia2025.02 | 72 | 76.9 | 630.4 | |
| Pythia-160MNumber of trainable vectors=1, Backbone family=Pythia2025.02 | 65 | 61.3 | 500.8 |