Compression Capacity on Fanfics (AO3) published after October 2024 (test)
1,568Max TokensLlama-3.1-8B
Evaluation Results
| Method | Links | |||
|---|---|---|---|---|
| Llama-3.1-8BNumber of trainable vectors=1, Backbone family=Llama2025.02 | 1,568 | 1,071.8 | 4,768.9 | |
| Llama-3.2-3BNumber of trainable vectors=1, Backbone family=Llama2025.02 | 1,024 | 734.1 | 3,354.5 | |
| Llama-3.2-1BNumber of trainable vectors=1, Backbone family=Llama2025.02 | 512 | 449.6 | 2,213.8 | |
| Pythia-1.4BNumber of trainable vectors=1, Backbone family=Pythia2025.02 | 192 | 152.9 | 776.9 | |
| Pythia-410MNumber of trainable vectors=1, Backbone family=Pythia2025.02 | 96 | 81.2 | 429.8 | |
| Pythia-160MNumber of trainable vectors=1, Backbone family=Pythia2025.02 | 80 | 70.9 | 378.1 |