General Capability on MMLU
79.6MMLU AccuracyOriginal Weight Model
Evaluation results
| Method | Links | |
|---|---|---|
| Original Weight ModelModel=Qwen2.5 14B Instruct2025.04 | 79.6 | |
| REPBENDModel=Qwen2.5 14B Instruct2025.04 | 78.89 | |
| QWEN 3 14BModel=QWEN 3 14B2025.10 | 76.1 | |
| ELSModel=QWEN 3 14B2025.10 | 76.1 | |
| SurgicalModel=QWEN 3 14B2025.10 | 74.7 | |
| AdaSteerModel=QWEN 3 14B2025.10 | 74.4 | |
| CASTModel=QWEN 3 14B2025.10 | 74 | |
| AlphaSteerModel=QWEN 3 14B2025.10 | 73.3 | |
| QWEN 3 8BModel=QWEN 3 8B2025.10 | 72.8 | |
| ELSModel=QWEN 3 8B2025.10 | 72.8 | |
| AlphaSteerModel=QWEN 3 8B2025.10 | 72.3 | |
| SurgicalModel=QWEN 3 8B2025.10 | 71.2 | |
| AdaSteerModel=QWEN 3 8B2025.10 | 70.9 | |
| CASTModel=QWEN 3 8B2025.10 | 70.5 | |
| system promptModel=QWEN 3 8B2025.10 | 70.2 | |
| system promptModel=QWEN 3 14B2025.10 | 69.8 | |
| LLAMA3.1-8B-INSTModel=LLAMA3.1-8B-INST2025.10 | 68.1 | |
| ELSModel=LLAMA3.1-8B-INST2025.10 | 68.1 | |
| Llama-3.1-8B-ITModel Status=Base Model, Evaluation Metric Type=Acc2025.10 | 68 | |
| ELSBase Model=Llama-3.1-8B-IT, Evaluation Metric Type=Acc2025.10 | 68 | |
| SurgicalModel=LLAMA3.1-8B-INST2025.10 | 67.7 | |
| CASTModel=LLAMA3.1-8B-INST2025.10 | 67.3 | |
| AlphaSteerModel=LLAMA3.1-8B-INST2025.10 | 66.7 | |
| WHP (W/O safe set)Backbone=Llama3 8B Instruct2025.04 | 66.44 | |
| NPO (W/O safe set)Backbone=Llama3 8B Instruct2025.04 | 66.29 | |
| WHPBackbone=Llama3 8B Instruct2025.04 | 66.16 | |
| NPOBackbone=Llama3 8B Instruct2025.04 | 66.13 | |
| TABackbone=Llama3 8B Instruct2025.04 | 66.09 | |
| SFTBackbone=Llama3 8B Instruct2025.04 | 66.01 | |
| AdaSteerModel=LLAMA3.1-8B-INST2025.10 | 66 | |
| OriginalBackbone=Llama3 8B Instruct2025.04 | 65.89 | |
| CB*Backbone=Llama3 8B Instruct, publicly_available_safety_tuned=true2025.04 | 65.77 | |
| CBBackbone=Llama3 8B Instruct2025.04 | 65.77 | |
| DPOBackbone=Llama3 8B Instruct2025.04 | 65.31 | |
| REPBENDBackbone=Llama3 8B Instruct2025.04 | 65.08 | |
| Safety PromptingBackbone=Llama3 8B Instruct2025.04 | 64.41 | |
| SFTBase Model=Llama-3.1-8B-IT, Evaluation Metric Type=Acc2025.10 | 63.4 | |
| Self-PlayBase Model=Llama-3.1-8B-IT, Evaluation Metric Type=Acc2025.10 | 62.4 | |
| Defender-Only + SFTBase Model=Llama-3.1-8B-IT, Evaluation Metric Type=Acc2025.10 | 62.3 | |
| Self-Play + SFTBase Model=Llama-3.1-8B-IT, Evaluation Metric Type=Acc2025.10 | 62.3 | |
| Block-R1Source Domain=Multi-domain, Backbone=LLaDA-8B-Instruct2026.05 | 62.22 | |
| Defender-OnlyBase Model=Llama-3.1-8B-IT, Evaluation Metric Type=Acc2025.10 | 62.2 | |
| system promptModel=LLAMA3.1-8B-INST2025.10 | 62 | |
| OriginalBackbone=Mistral 7B Instruct-v0.22025.04 | 60.18 | |
| WHP (W/O safe set)Backbone=Mistral 7B Instruct-v0.22025.04 | 60.08 | |
| CB*Backbone=Mistral 7B Instruct-v0.2, publicly_available_safety_tuned=true2025.04 | 60.08 | |
| CBBackbone=Mistral 7B Instruct-v0.22025.04 | 59.93 | |
| NPOBackbone=Mistral 7B Instruct-v0.22025.04 | 59.61 | |
| REPBENDBackbone=Mistral 7B Instruct-v0.22025.04 | 59.48 | |
| R2D2*Backbone=Mistral 7B Instruct-v0.2, publicly_available_safety_tuned=true2025.04 | 59.44 | |
| Safety PromptingBackbone=Mistral 7B Instruct-v0.22025.04 | 59.26 | |
| WHPBackbone=Mistral 7B Instruct-v0.22025.04 | 59.24 | |
| DPOBackbone=Mistral 7B Instruct-v0.22025.04 | 59.21 | |
| NPO (W/O safe set)Backbone=Mistral 7B Instruct-v0.22025.04 | 59.12 | |
| TABackbone=Mistral 7B Instruct-v0.22025.04 | 58.57 | |
| SFTBackbone=Mistral 7B Instruct-v0.22025.04 | 58.46 | |
| RMUBackbone=Llama3 8B Instruct2025.04 | 58.23 | |
| REPBENDModel=Gemma2 2B Instruct2025.04 | 58.14 | |
| Original Weight ModelModel=Gemma2 2B Instruct2025.04 | 57.98 | |
| QWEN 3 1.7BModel=QWEN 3 1.7B2025.10 | 57.9 | |
| ELSModel=QWEN 3 1.7B2025.10 | 57.9 | |
| AdaSteerModel=QWEN 3 1.7B2025.10 | 57.4 | |
| SurgicalModel=QWEN 3 1.7B2025.10 | 57.2 | |
| CASTModel=QWEN 3 1.7B2025.10 | 56.8 | |
| AlphaSteerModel=QWEN 3 1.7B2025.10 | 56.8 | |
| RMUBackbone=Mistral 7B Instruct-v0.22025.04 | 50.02 | |
| system promptModel=QWEN 3 1.7B2025.10 | 49.1 | |
| LLAMA2-7B-CHATModel=LLAMA2-7B-CHAT2025.10 | 47.6 | |
| ELSModel=LLAMA2-7B-CHAT2025.10 | 47.6 | |
| system promptModel=LLAMA2-7B-CHAT2025.10 | 47.5 | |
| SurgicalModel=LLAMA2-7B-CHAT2025.10 | 47 | |
| AdaSteerModel=LLAMA2-7B-CHAT2025.10 | 46 | |
| AlphaSteerModel=LLAMA2-7B-CHAT2025.10 | 46 | |
| CASTModel=LLAMA2-7B-CHAT2025.10 | 45.6 |