Cross-domain Model Evaluation on Aggregate Downstream Suite
64.16Average ScoreDense baseline
Evaluation results
| Method | Links | ||
|---|---|---|---|
| Dense baselineBase Model=Llama-3.1-8B-Instruct, Sparsity=Dense2026.06 | 64.16 | 100 | |
| SUBFITBase Model=Llama-3.1-8B-Instruct, Sparsity=12.5%2026.06 | 61.4 | 95.7 | |
| Dense baselineBase Model=Llama-3.2-3B-Instruct, Sparsity=Dense2026.06 | 57.87 | 100 | |
| SUBFITBase Model=Llama-3.2-3B-Instruct, Sparsity=12.5%2026.06 | 54.98 | 95 | |
| ReplaceMe (Cosine)Base Model=Llama-3.2-3B-Instruct, Sparsity=12.5%2026.06 | 54.57 | 94.3 | |
| ReplaceMe (LS)Base Model=Llama-3.2-3B-Instruct, Sparsity=12.5%2026.06 | 54.32 | 93.86 | |
| Streamline (Layer)Base Model=Llama-3.2-3B-Instruct, Sparsity=12.5%2026.06 | 54.04 | 93.38 | |
| Streamline (FFN)Base Model=Llama-3.2-3B-Instruct, Sparsity=12.5%2026.06 | 53.45 | 92.36 |