Utility Evaluation on MMLU and GSM8K
70.4MMLU AccuracyPALETTE
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| PALETTEAllowed Domain=Violence, Backbone=QWEN2.5-7B-INSTRUCT2026.05 | 70.4 | 85.3 | 77.9 | |
| BaseAllowed Domain=None, Backbone=QWEN2.5-7B-INSTRUCT2026.05 | 70.1 | 86.5 | 78.3 | |
| PALETTEAllowed Domain=Sexual, Backbone=QWEN2.5-7B-INSTRUCT2026.05 | 70 | 86.2 | 78.1 | |
| PALETTEAllowed Domain=Disinfo, Backbone=QWEN2.5-7B-INSTRUCT2026.05 | 69.9 | 85.1 | 77.5 | |
| PALETTEAllowed Domain=Illegal, Backbone=QWEN2.5-7B-INSTRUCT2026.05 | 69.9 | 85.6 | 77.8 | |
| PALETTEAllowed Domain=Hate, Backbone=QWEN2.5-7B-INSTRUCT2026.05 | 69.7 | 86.1 | 77.9 | |
| CASTAllowed Domain=Illegal, Backbone=QWEN2.5-7B-INSTRUCT2026.05 | 68.3 | 81.7 | 75 | |
| CASTAllowed Domain=Hate, Backbone=QWEN2.5-7B-INSTRUCT2026.05 | 67.5 | 82.4 | 75 | |
| CASTAllowed Domain=Sexual, Backbone=QWEN2.5-7B-INSTRUCT2026.05 | 66.8 | 80.4 | 73.6 | |
| CASTAllowed Domain=Violence, Backbone=QWEN2.5-7B-INSTRUCT2026.05 | 66.4 | 81.7 | 74.1 | |
| CASTAllowed Domain=Disinfo, Backbone=QWEN2.5-7B-INSTRUCT2026.05 | 64.8 | 79.7 | 72.3 | |
| AutoDANAllowed Domain=Sexual, Backbone=QWEN2.5-7B-INSTRUCT2026.05 | 56.9 | 63.7 | 60.3 | |
| AutoDANAllowed Domain=Illegal, Backbone=QWEN2.5-7B-INSTRUCT2026.05 | 56.9 | 63.7 | 60.3 | |
| AutoDANAllowed Domain=Violence, Backbone=QWEN2.5-7B-INSTRUCT2026.05 | 49.3 | 59.1 | 54.2 | |
| AutoDANAllowed Domain=Hate, Backbone=QWEN2.5-7B-INSTRUCT2026.05 | 49.3 | 59.1 | 54.2 | |
| AutoDANAllowed Domain=Disinfo, Backbone=QWEN2.5-7B-INSTRUCT2026.05 | 49.3 | 59.1 | 54.2 |