Causal intervention on Gemma-2-2B (layer 12)
11.72SCRMatryoshka
Evaluation Results
| Method | Links | ||
|---|---|---|---|
| MatryoshkaBackbone=Gemma-2-2B, Layer=12, k (sparsity)=1002026.06 | 11.72 | 3.7 | |
| C2RBackbone=Gemma-2-2B, Layer=12, k (sparsity)=1002026.06 | 10.47 | 2.26 | |
| OrtBackbone=Gemma-2-2B, Layer=12, k (sparsity)=1002026.06 | 8.59 | 0.61 | |
| TopKBackbone=Gemma-2-2B, Layer=12, k (sparsity)=1002026.06 | 4.81 | 0.38 | |
| Batch TopKBackbone=Gemma-2-2B, Layer=12, k (sparsity)=1002026.06 | 3.53 | 0.45 |