Indirect Object Identification on IOI
0.15Last-Token KL DivergenceFAP
Evaluation Results
| Method | Links | |
|---|---|---|
| FAPModel=Gemma-2-2B, Budget (K)=8002026.04 | 0.15 | |
| FAP-SynergyModel=Gemma-2-2B, Budget (K)=8002026.04 | 0.15 | |
| FAPModel=Llama-3.2-1B, Budget (K)=8002026.04 | 0.22 | |
| FAP-SynergyModel=Llama-3.2-1B, Budget (K)=8002026.04 | 0.22 | |
| FAPModel=Gemma-2-2B, Budget (K)=4002026.04 | 0.3 | |
| FAP-SynergyModel=Gemma-2-2B, Budget (K)=4002026.04 | 0.3 | |
| FAPModel=Gemma-2-2B, Budget (K)=2002026.04 | 0.52 | |
| FAP-SynergyModel=Gemma-2-2B, Budget (K)=2002026.04 | 0.52 | |
| FAPModel=Llama-3.2-1B, Budget (K)=4002026.04 | 0.53 | |
| FAP-SynergyModel=Llama-3.2-1B, Budget (K)=4002026.04 | 0.53 | |
| ACDC-Style PruningModel=Gemma-2-2B, Budget (K)=8002026.04 | 0.64 | |
| Activation-MagnitudeModel=Gemma-2-2B, Budget (K)=8002026.04 | 0.7 | |
| ACDC-Style PruningModel=Gemma-2-2B, Budget (K)=4002026.04 | 0.72 | |
| FAP-SynergyModel=Gemma-2-2B, Budget (K)=1002026.04 | 0.73 | |
| FAPModel=Gemma-2-2B, Budget (K)=1002026.04 | 0.74 | |
| ACDC-Style PruningModel=Gemma-2-2B, Budget (K)=2002026.04 | 0.76 | |
| ACDC-Style PruningModel=Gemma-2-2B, Budget (K)=1002026.04 | 0.77 | |
| ACDC-Style PruningModel=Gemma-2-2B, Budget (K)=502026.04 | 0.81 | |
| FAP-SynergyModel=Gemma-2-2B, Budget (K)=502026.04 | 0.82 | |
| FAPModel=Llama-3.2-1B, Budget (K)=2002026.04 | 0.85 | |
| FAP-SynergyModel=Llama-3.2-1B, Budget (K)=2002026.04 | 0.85 | |
| Activation-MagnitudeModel=Llama-3.2-1B, Budget (K)=8002026.04 | 0.87 | |
| FAPModel=Gemma-2-2B, Budget (K)=502026.04 | 0.91 | |
| Activation-MagnitudeModel=Gemma-2-2B, Budget (K)=4002026.04 | 0.96 | |
| Activation-MagnitudeModel=Llama-3.2-1B, Budget (K)=4002026.04 | 1.07 | |
| FAP-SynergyModel=Llama-3.2-1B, Budget (K)=1002026.04 | 1.12 | |
| FAPModel=Llama-3.2-1B, Budget (K)=1002026.04 | 1.13 | |
| ACDC-Style PruningModel=Llama-3.2-1B, Budget (K)=8002026.04 | 1.13 | |
| Activation-MagnitudeModel=Gemma-2-2B, Budget (K)=2002026.04 | 1.18 | |
| ACDC-Style PruningModel=Llama-3.2-1B, Budget (K)=4002026.04 | 1.19 | |
| FAP-SynergyModel=Llama-3.2-1B, Budget (K)=502026.04 | 1.22 | |
| ACDC-Style PruningModel=Llama-3.2-1B, Budget (K)=2002026.04 | 1.22 | |
| ACDC-Style PruningModel=Llama-3.2-1B, Budget (K)=1002026.04 | 1.24 | |
| Activation-MagnitudeModel=Gemma-2-2B, Budget (K)=1002026.04 | 1.25 | |
| ACDC-Style PruningModel=Llama-3.2-1B, Budget (K)=502026.04 | 1.26 | |
| Activation-MagnitudeModel=Gemma-2-2B, Budget (K)=502026.04 | 1.29 | |
| Activation-MagnitudeModel=Llama-3.2-1B, Budget (K)=2002026.04 | 1.32 | |
| FAPModel=Llama-3.2-1B, Budget (K)=502026.04 | 1.33 | |
| Activation-MagnitudeModel=Llama-3.2-1B, Budget (K)=1002026.04 | 1.52 | |
| Activation-MagnitudeModel=Llama-3.2-1B, Budget (K)=502026.04 | 1.59 |