Multiple-Choice Question Answering on MMLU Humanities (test)
62.47Accuracy (Humanities)Knowledge Probe (KP)
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| Knowledge Probe (KP)Model=Qwen 2.5 7B2025.09 | 62.47 | 100 | 0 | |
| KAPPAModel=Qwen 2.5 7B, Intervention Layers=62025.09 | 61.92 | 85.22 | 0.703 | |
| KAPPAModel=Qwen 2.5 7B, Intervention Layers=32025.09 | 61.04 | 82.23 | 0.72 | |
| KAPPAModel=Qwen 2.5 7B, Intervention Layers=12025.09 | 60.77 | 82.03 | 0.751 | |
| KAPPAModel=Llama 3.1 8B, Intervention Layers=62025.09 | 60.73 | 87 | 0.446 | |
| KAPPAModel=Llama 3.1 8B, Intervention Layers=12025.09 | 60.67 | 87.15 | 0.533 | |
| KAPPAModel=Llama 3.1 8B, Intervention Layers=32025.09 | 60.32 | 85.19 | 0.456 | |
| CAAModel=Qwen 2.5 7B2025.09 | 59.92 | 78.42 | 0.777 | |
| BaseModel=Qwen 2.5 7B2025.09 | 59.91 | 78.5 | 0.777 | |
| DoLAModel=Qwen 2.5 7B2025.09 | 59.85 | 78.64 | 0.811 | |
| CAAModel=Llama 3.1 8B2025.09 | 59.62 | 77.56 | 0.487 | |
| BaseModel=Llama 3.1 8B2025.09 | 58.57 | 77.94 | 0.469 | |
| DoLAModel=Llama 3.1 8B2025.09 | 56.08 | 69.35 | 0.504 |