Attention Head Selection (Causal Tracing) on WinoBias benchmark leaderboard · SOTA2 Research

Attention Head Selection (Causal Tracing) on WinoBias

0.0128ℓ(D, m) (10% Sparsity)random

Evaluation Results

Method	Links
random2026.06		0.0128	0.0146	0.0146	0.016	6.55
top-k2026.06		0.0157	0.0171	0.0178	0.0181	0.47
greedy2026.06		0.0159	0.0176	0.0186	0.0192	10.62
PGB-CT2026.06		0.0159	0.0174	0.0181	0.019	1.44
random2026.06		0.064	0.073	0.077	0.085	75.38
random2026.06		0.08	0.096	0.1	0.103	10.25
top-k2026.06		0.089	0.09	0.091	0.092	31.97
PGB-CT2026.06		0.09	0.094	0.096	0.097	9.39
top-k2026.06		0.108	0.111	0.113	0.114	1.49
PGB-CT2026.06		0.109	0.115	0.117	0.119	2.61
greedy2026.06		0.111	0.116	0.119	0.12	68.42
random2026.06		0.164	0.216	0.262	0.297	36.8
random2026.06		0.19	0.247	0.305	0.336	21.45
random2026.06		0.221	0.393	0.442	0.46	64.98
top-k2026.06		0.279	0.358	0.386	0.415	26.57
random2026.06		0.322	0.326	0.325	0.351	51.16
top-k2026.06		0.353	0.357	0.361	0.364	26.07
PGB-CT2026.06		0.356	0.37	0.375	0.383	8.1
top-k2026.06		0.374	0.378	0.389	0.388	8.18
PGB-CT2026.06		0.381	0.394	0.401	0.404	5.32
greedy2026.06		0.391	0.406	0.415	0.42	1,001.5
top-k2026.06		0.523	0.531	0.536	0.539	62.85
PGB-CT2026.06		0.546	0.563	0.567	0.576	11.32
PGB-CT2026.06		0.752	1.007	1.766	2.747	5.82