Arithmetic Reasoning on Two-operand Arithmetic Reasoning (test)
84.33Addition AccuracyRASTEER
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| RASTEERBackbone=Llama3-8b, Top-k attention heads=30, Sorting criterion=recall2025.06 | 84.33 | 88 | 80.08 | 79.91 | |
| Llama3-8bSteering=Baseline2025.06 | 82.33 | 87.91 | 74.08 | 78.16 | |
| RASTEERBackbone=Pythia-6.9b, Top-k attention heads=30, Sorting criterion=recall2025.06 | 31.66 | 2.5 | 34.66 | 34.75 | |
| Pythia-6.9bSteering=Baseline2025.06 | 25.41 | 0 | 14.41 | 19.91 | |
| GPT-2 XLSteering=Baseline2025.06 | 0 | 0 | 17.5 | 19.66 | |
| RASTEERBackbone=GPT-2 XL, Top-k attention heads=30, Sorting criterion=recall2025.06 | 0 | 0 | 17.5 | 23.66 |