Utility on MMLU
83.1AccuracyGPT-OSS-20B
Evaluation results
| Method | Links | |
|---|---|---|
| GPT-OSS-20BSetting=Baseline2026.05 | 83.1 | |
| Qwen3-8BSetting=Baseline2026.05 | 80.3 | |
| Qwen3-8BSetting=BCT2026.05 | 78.9 | |
| Qwen3-8BSetting=ACT2026.05 | 76.8 | |
| Phi-4-reasoningSetting=Baseline2026.05 | 75.9 | |
| Qwen (Base)LLM=Qwen, SFT Status=None2025.05 | 75.78 | |
| GPT-OSS-20BSetting=ACT2026.05 | 74.9 | |
| Phi-4-reasoningSetting=ACT2026.05 | 73.3 | |
| GPT-OSS-20BSetting=BCT2026.05 | 72.4 | |
| Gemma-4-E4B-itSetting=Baseline2026.05 | 69.6 | |
| Gemma-4-E4B-itSetting=BCT2026.05 | 68.3 | |
| Gemma-4-E4B-itSetting=ACT2026.05 | 67.8 | |
| Qwen3-1.7BSetting=Baseline2026.05 | 60.9 | |
| Qwen (SFT_DB)LLM=Qwen, SFT Status=SFT on AIDSAFE-generated CoTs and responses2025.05 | 60.52 | |
| Qwen3-1.7BSetting=BCT2026.05 | 59.7 | |
| Qwen3-1.7BSetting=ACT2026.05 | 58.9 | |
| Phi-4-reasoningSetting=BCT2026.05 | 58.1 | |
| Qwen (SFT_OG)LLM=Qwen, SFT Status=SFT on original response data without CoTs2025.05 | 55.73 | |
| Mixtral (Base)LLM=Mixtral, SFT Status=None2025.05 | 35.42 | |
| Mixtral (SFT_DB)LLM=Mixtral, SFT Status=SFT on AIDSAFE-generated CoTs and responses2025.05 | 34.51 | |
| SFT_DBTraining=Supervised Fine-Tuning, Data Generation=AIDSAFE2025.05 | 34.51 | |
| DPO_WhispererAlignment=DPO, Data Recipe=ear-whisperer, Base Model=SFT_DB2025.05 | 33.07 | |
| Mixtral (SFT_OG)LLM=Mixtral, SFT Status=SFT on original response data without CoTs2025.05 | 31.38 |