Bias Evaluation on BBQ Religion (test)
80.5Accuracy (ACC)KAPPA
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| KAPPAModel=Qwen 2.5 7B, Intervention layers=62025.09 | 80.5 | — | 98.8 | 0.55 | |
| KPModel=Qwen 2.5 7B2025.09 | 80.3 | — | 100 | 0 | |
| KAPPAModel=Qwen 2.5 7B, Intervention layers=32025.09 | 80.2 | — | 99 | 0.55 | |
| KAPPAModel=Qwen 2.5 7B, Intervention layers=12025.09 | 80.1 | — | 99.4 | 0.55 | |
| Qwen 2.5 7B Instructk=32025.09 | 79.6 | 0.7 | 98.5 | 0.54 | |
| BaseModel=Qwen 2.5 7B2025.09 | 79.6 | — | 98.5 | 0.54 | |
| CAAModel=Qwen 2.5 7B2025.09 | 79.5 | — | 98.5 | 0.54 | |
| DoLAModel=Qwen 2.5 7B2025.09 | 79.5 | — | 98.2 | 0.54 | |
| KPModel=Llama 3.1 8B2025.09 | 78.1 | — | 100 | 0 | |
| KAPPAModel=Llama 3.1 8B, Intervention layers=32025.09 | 76.5 | — | 92.9 | 0.41 | |
| KAPPAModel=Llama 3.1 8B, Intervention layers=12025.09 | 75.7 | — | 93.6 | 0.46 | |
| CAAModel=Llama 3.1 8B2025.09 | 68.8 | — | 80 | 0.42 | |
| KAPPAModel=Llama 3.1 8B, Intervention layers=62025.09 | 68.8 | — | 81.6 | 0.57 | |
| Llama 3.1 8B Instructk=32025.09 | 67.6 | 10.6 | 79.1 | 0.42 | |
| BaseModel=Llama 3.1 8B2025.09 | 67.6 | — | 79.1 | 0.42 | |
| DoLAModel=Llama 3.1 8B2025.09 | 64.9 | — | 76.6 | 0.42 |