Multiple-Choice Question Answering on MMLU Social Sciences (test)
79.12AccuracyKAPPA
Evaluation Results
| Method | Links | |||
|---|---|---|---|---|
| KAPPAModel=Qwen 2.5 7B, Intervention Layers=62025.09 | 79.12 | 95.47 | 0.759 | |
| Knowledge Probe (KP)Model=Qwen 2.5 7B2025.09 | 79.02 | 100 | 0 | |
| KAPPAModel=Qwen 2.5 7B, Intervention Layers=32025.09 | 78.95 | 96.78 | 0.739 | |
| KAPPAModel=Qwen 2.5 7B, Intervention Layers=12025.09 | 78.91 | 97.12 | 0.734 | |
| BaseModel=Qwen 2.5 7B2025.09 | 78.85 | 95.93 | 0.718 | |
| CAAModel=Qwen 2.5 7B2025.09 | 78.81 | 95.94 | 0.718 | |
| DoLAModel=Qwen 2.5 7B2025.09 | 78.52 | 94.81 | 0.725 | |
| BaseModel=Llama 3.1 8B2025.09 | 74.04 | 90.55 | 0.471 | |
| CAAModel=Llama 3.1 8B2025.09 | 73.93 | 90.62 | 0.466 | |
| KAPPAModel=Llama 3.1 8B, Intervention Layers=12025.09 | 73.79 | 96.08 | 0.617 | |
| KAPPAModel=Llama 3.1 8B, Intervention Layers=32025.09 | 73.24 | 94.54 | 0.607 | |
| DoLAModel=Llama 3.1 8B2025.09 | 72.25 | 84 | 0.571 | |
| KAPPAModel=Llama 3.1 8B, Intervention Layers=62025.09 | 70.49 | 85.98 | 0.778 |