List your company

Attention Head Selection (Causal Tracing) on WinoGender

0.0082Loss (10% Sparsity)random
-0.0094720.1098140.22910.348386Jun 2, 2026

Evaluation results

MethodLinks
randomModel=DistilGPT2, Selection Algorithm=random2026.060.00820.00930.00910.012.23
top-kModel=DistilGPT2, Selection Algorithm=top-k2026.060.010.01170.01230.01280.18
greedyModel=DistilGPT2, Selection Algorithm=greedy2026.060.01030.01220.01340.0143.57
PGB-CTModel=DistilGPT2, Selection Algorithm=PGB-CT2026.060.01030.01180.0130.01370.43
randomModel=GPT2-small, Selection Algorithm=random2026.060.0270.0320.0360.0373.54
randomModel=Qwen3-1.7B, Selection Algorithm=random2026.060.0320.0350.0410.03829.36
top-kModel=GPT2-small, Selection Algorithm=top-k2026.060.0370.0410.0420.0440.5
PGB-CTModel=GPT2-small, Selection Algorithm=PGB-CT2026.060.0390.0440.0480.0490.79
greedyModel=GPT2-small, Selection Algorithm=greedy2026.060.040.0450.0490.0523.03
top-kModel=Qwen3-1.7B, Selection Algorithm=top-k2026.060.0410.0420.0450.04413.01
PGB-CTModel=Qwen3-1.7B, Selection Algorithm=PGB-CT2026.060.0420.0460.0480.053.21
randomModel=GPT2-xl, Selection Algorithm=random2026.060.0470.0780.0940.09824.83
randomModel=GPT2-large, Selection Algorithm=random2026.060.0850.0990.1110.12813.43
randomModel=GPT2-medium, Selection Algorithm=random2026.060.1380.1530.1650.1737.23
top-kModel=GPT2-large, Selection Algorithm=top-k2026.060.1460.1590.1650.1689.46
PGB-CTModel=GPT2-large, Selection Algorithm=PGB-CT2026.060.1520.1710.1840.1922.43
top-kModel=GPT2-xl, Selection Algorithm=top-k2026.060.1550.1750.1860.18829.38
PGB-CTModel=GPT2-xl, Selection Algorithm=PGB-CT2026.060.1590.190.2050.2123.45
top-kModel=GPT2-medium, Selection Algorithm=top-k2026.060.1910.2010.2030.2052.76
PGB-CTModel=GPT2-medium, Selection Algorithm=PGB-CT2026.060.2030.2180.2270.2331.56
greedyModel=GPT2-medium, Selection Algorithm=greedy2026.060.2080.2240.2320.237357.28
randomModel=Llama-3.2-1B, Selection Algorithm=random2026.060.3260.3830.3870.41820.71
PGB-CTModel=Llama-3.2-1B, Selection Algorithm=PGB-CT2026.060.3660.4440.4772.4491.9
top-kModel=Llama-3.2-1B, Selection Algorithm=top-k2026.060.450.4160.40.36610.31