Natural Language Processing Reasoning on BBH-NLP (test)
66.2Accuracy (ACC)KP
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| KPModel=Qwen 2.5 7B2025.09 | 66.2 | — | 100 | 0 | |
| KAPPAModel=Qwen 2.5 7B, Intervention layers=62025.09 | 63.6 | — | 74.9 | 0.87 | |
| KPModel=Llama 3.1 8B2025.09 | 63.5 | — | 100 | 0 | |
| KAPPAModel=Qwen 2.5 7B, Intervention layers=12025.09 | 63 | — | 74 | 0.89 | |
| KAPPAModel=Llama 3.1 8B, Intervention layers=32025.09 | 63 | — | 83.7 | 0.39 | |
| KAPPAModel=Qwen 2.5 7B, Intervention layers=32025.09 | 62.8 | — | 73.2 | 0.89 | |
| KAPPAModel=Llama 3.1 8B, Intervention layers=12025.09 | 62.4 | — | 86.3 | 0.47 | |
| CAAModel=Qwen 2.5 7B2025.09 | 61.2 | — | 69.9 | 0.92 | |
| Qwen 2.5 7B Instructk=42025.09 | 61.1 | 5.1 | 69.8 | 0.92 | |
| BaseModel=Qwen 2.5 7B2025.09 | 61.1 | — | 69.8 | 0.92 | |
| DoLAModel=Qwen 2.5 7B2025.09 | 61 | — | 69.3 | 0.92 | |
| CAAModel=Llama 3.1 8B2025.09 | 60.2 | — | 73.3 | 0.41 | |
| KAPPAModel=Llama 3.1 8B, Intervention layers=62025.09 | 60.2 | — | 75.3 | 0.46 | |
| Llama 3.1 8B Instructk=42025.09 | 59.6 | 3.9 | 73.9 | 0.41 | |
| BaseModel=Llama 3.1 8B2025.09 | 59.6 | — | 73.9 | 0.41 | |
| DoLAModel=Llama 3.1 8B2025.09 | 56.6 | — | 69.6 | 0.51 |