Algorithmic Reasoning on BBH-Algorithmic (test)
55.4ACCKP
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| KPModel=Qwen 2.5 7B2025.09 | 55.4 | — | 100 | 0 | |
| KAPPAModel=Qwen 2.5 7B, Intervention layers=62025.09 | 53.6 | — | 78.9 | 0.66 | |
| KAPPAModel=Qwen 2.5 7B, Intervention layers=32025.09 | 52.2 | — | 75.3 | 0.65 | |
| KAPPAModel=Qwen 2.5 7B, Intervention layers=12025.09 | 51.5 | — | 72.5 | 0.69 | |
| CAAModel=Qwen 2.5 7B2025.09 | 51.2 | — | 69.8 | 0.71 | |
| Qwen 2.5 7B Instructk=42025.09 | 51 | 4.4 | 69.6 | 0.7 | |
| BaseModel=Qwen 2.5 7B2025.09 | 51 | — | 69.6 | 0.7 | |
| KPModel=Llama 3.1 8B2025.09 | 50.7 | — | 100 | 0 | |
| DoLAModel=Qwen 2.5 7B2025.09 | 50.4 | — | 67.5 | 0.75 | |
| KAPPAModel=Llama 3.1 8B, Intervention layers=62025.09 | 50.1 | — | — | 0.3 | |
| KAPPAModel=Llama 3.1 8B, Intervention layers=32025.09 | 49.5 | — | 82.9 | 0.26 | |
| KAPPAModel=Llama 3.1 8B, Intervention layers=12025.09 | 49.3 | — | 83.1 | 0.31 | |
| Llama 3.1 8B Instructk=42025.09 | 45.1 | 5.5 | 62.1 | 0.23 | |
| BaseModel=Llama 3.1 8B2025.09 | 45.1 | — | 62.1 | 0.23 | |
| CAAModel=Llama 3.1 8B2025.09 | 45.1 | — | 62.1 | 0.26 | |
| DoLAModel=Llama 3.1 8B2025.09 | 42.4 | — | 47.4 | 0.29 |