Tool Activation Probing on τ-bench airline (80/20 stratified split)
89Top-1 AccuracyQwen 3 14B
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| Qwen 3 14BK=5, CHANCE=20.0%, Probe Type=multinomial logistic-regression, Feature Source=residual stream, Regularization=l2, C=12026.05 | 89 | 100 | 4.5 | |
| Qwen 3 8BK=6, CHANCE=16.7%, Probe Type=multinomial logistic-regression, Feature Source=residual stream, Regularization=l2, C=12026.05 | 72 | 100 | 4.4 | |
| Qwen 3 4BK=5, CHANCE=20.0%, Probe Type=multinomial logistic-regression, Feature Source=residual stream, Regularization=l2, C=12026.05 | 71 | 100 | 3.5 | |
| Gemma 3 12B ITK=9, CHANCE=11.1%, Probe Type=multinomial logistic-regression, Feature Source=residual stream, Regularization=l2, C=12026.05 | 66 | 97 | 5.9 | |
| Gemma 3 4B ITK=8, CHANCE=12.5%, Probe Type=multinomial logistic-regression, Feature Source=residual stream, Regularization=l2, C=12026.05 | 61 | 96 | 4.9 |