Multiple-Choice Question Answering on MMLU STEM (test)
65.6Accuracy (ACC)DoLA
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| DoLAModel=Qwen 2.5 7B2025.09 | 65.6 | 87.81 | 0.714 | |
| KAPPAModel=Qwen 2.5 7B, Intervention Layers=62025.09 | 65.56 | 91.99 | 0.764 | |
| KAPPAModel=Qwen 2.5 7B, Intervention Layers=32025.09 | 65.45 | 92.7 | 0.724 | |
| Knowledge Probe (KP)Model=Qwen 2.5 7B2025.09 | 65.36 | 100 | 0 | |
| KAPPAModel=Qwen 2.5 7B, Intervention Layers=12025.09 | 65.24 | 93.17 | 0.721 | |
| CAAModel=Qwen 2.5 7B2025.09 | 65.19 | 89.5 | 0.708 | |
| BaseModel=Qwen 2.5 7B2025.09 | 65.17 | 89.7 | 0.703 | |
| CAAModel=Llama 3.1 8B2025.09 | 54.85 | 90.09 | 0.344 | |
| BaseModel=Llama 3.1 8B2025.09 | 54.65 | 91.12 | 0.321 | |
| KAPPAModel=Llama 3.1 8B, Intervention Layers=12025.09 | 54.34 | 90.56 | 0.684 | |
| KAPPAModel=Llama 3.1 8B, Intervention Layers=62025.09 | 54.23 | 84.58 | 0.899 | |
| KAPPAModel=Llama 3.1 8B, Intervention Layers=32025.09 | 54.03 | 86.3 | 0.755 | |
| DoLAModel=Llama 3.1 8B2025.09 | 51.76 | 71.36 | 0.533 |