Over-Refusal Attack Resistance Evaluation on Over Refusal
65.57MMLUQ-Misalign
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| Q-MisalignQuantization=FP32, Model=Qwen2.5-3B-Instruct, Zero-shot=true2026.01 | 65.57 | 53.14 | 133 | |
| OriginalQuantization=FP32, Model=Qwen2.5-3B-Instruct, Zero-shot=true2026.01 | 65.5 | 58.69 | 67 | |
| OriginalQuantization=BF16, Model=Qwen2.5-3B-Instruct, Zero-shot=true2026.01 | 65.48 | 58.68 | 67 | |
| ELQQuantization=FP32, Model=Qwen2.5-3B-Instruct, Zero-shot=true2026.01 | 65.43 | 54.9 | 133 | |
| ELQQuantization=BF16, Model=Qwen2.5-3B-Instruct, Zero-shot=true2026.01 | 65.42 | 54.95 | 0 | |
| Q-MisalignQuantization=BF16, Model=Qwen2.5-3B-Instruct, Zero-shot=true2026.01 | 65.4 | 53.3 | 67 | |
| ACLQuantization=FP32, Model=Qwen2.5-3B-Instruct, Zero-shot=true2026.01 | 64.91 | 58.05 | 133 | |
| OriginalQuantization=INT8, Model=Qwen2.5-3B-Instruct, Zero-shot=true2026.01 | 64.76 | 56.42 | 133 | |
| ACLQuantization=BF16, Model=Qwen2.5-3B-Instruct, Zero-shot=true2026.01 | 64.75 | 58.06 | 200 | |
| OriginalQuantization=NF4, Model=Qwen2.5-3B-Instruct, Zero-shot=true2026.01 | 64.34 | 56.83 | 133 | |
| ELQQuantization=INT8, Model=Qwen2.5-3B-Instruct, Zero-shot=true2026.01 | 64.02 | 56.12 | 3,800 | |
| Q-MisalignQuantization=INT8, Model=Qwen2.5-3B-Instruct, Zero-shot=true2026.01 | 63.72 | 56.03 | 3,267 | |
| ELQQuantization=NF4, Model=Qwen2.5-3B-Instruct, Zero-shot=true2026.01 | 63.27 | 56.29 | 3,533 | |
| Q-MisalignQuantization=NF4, Model=Qwen2.5-3B-Instruct, Zero-shot=true2026.01 | 63.16 | 56.34 | 3,467 | |
| OriginalQuantization=FP32, Model=Llama-3.2-3B-Instruct, Zero-Shot=true2026.01 | 62.3 | 51.47 | 2 | |
| OriginalQuantization=INT8, Model=Llama-3.2-3B-Instruct, Zero-Shot=true2026.01 | 62.21 | 51.74 | 0 | |
| OriginalQuantization=BF16, Model=Llama-3.2-3B-Instruct, Zero-Shot=true2026.01 | 62.18 | 51.46 | 0.67 | |
| ACLQuantization=INT8, Model=Qwen2.5-3B-Instruct, Zero-shot=true2026.01 | 61.11 | 59.83 | 6,800 | |
| ACLQuantization=NF4, Model=Qwen2.5-3B-Instruct, Zero-shot=true2026.01 | 60.25 | 61.51 | 7,267 | |
| OriginalQuantization=NF4, Model=Llama-3.2-3B-Instruct, Zero-Shot=true2026.01 | 60.01 | 50.45 | 2 | |
| ELQQuantization=FP32, Model=Llama-3.2-3B-Instruct, Zero-Shot=true2026.01 | 59.94 | 49.12 | 0.67 | |
| ELQQuantization=BF16, Model=Llama-3.2-3B-Instruct, Zero-Shot=true2026.01 | 59.92 | 49.14 | 1.33 | |
| OriginalQuantization=FP4, Model=Llama-3.2-3B-Instruct, Zero-Shot=true2026.01 | 59.74 | 50.29 | 0.67 | |
| Q-MisalignQuantization=FP32, Model=Llama-3.2-3B-Instruct, Zero-Shot=true2026.01 | 59.74 | 50.36 | 0 | |
| Q-MisalignQuantization=BF16, Model=Llama-3.2-3B-Instruct, Zero-Shot=true2026.01 | 59.65 | 49.11 | 0 | |
| OriginalQuantization=FP4, Model=Qwen2.5-3B-Instruct, Zero-shot=true2026.01 | 59.36 | 55.72 | 400 | |
| ACLQuantization=BF16, Model=Llama-3.2-3B-Instruct, Zero-Shot=true2026.01 | 59.29 | 49.25 | 0 | |
| ELQQuantization=INT8, Model=Llama-3.2-3B-Instruct, Zero-Shot=true2026.01 | 58.58 | 54.74 | 18 | |
| Q-MisalignQuantization=INT8, Model=Llama-3.2-3B-Instruct, Zero-Shot=true2026.01 | 58.47 | 54.67 | 24.67 | |
| ACLQuantization=FP32, Model=Llama-3.2-3B-Instruct, Zero-Shot=true2026.01 | 58.4 | 49.19 | 0 | |
| Q-MisalignQuantization=FP4, Model=Qwen2.5-3B-Instruct, Zero-shot=true2026.01 | 58.23 | 55.64 | 4,267 | |
| ELQQuantization=FP4, Model=Qwen2.5-3B-Instruct, Zero-shot=true2026.01 | 58.15 | 55.58 | 4,067 | |
| ACLQuantization=INT8, Model=Llama-3.2-3B-Instruct, Zero-Shot=true2026.01 | 57.32 | 54.51 | 70 | |
| Q-MisalignQuantization=NF4, Model=Llama-3.2-3B-Instruct, Zero-Shot=true2026.01 | 57.13 | 53.72 | 17.33 | |
| ELQQuantization=NF4, Model=Llama-3.2-3B-Instruct, Zero-Shot=true2026.01 | 57.12 | 53.71 | 25.33 | |
| ACLQuantization=NF4, Model=Llama-3.2-3B-Instruct, Zero-Shot=true2026.01 | 56.2 | 50.8 | 72.67 | |
| ACLQuantization=FP4, Model=Llama-3.2-3B-Instruct, Zero-Shot=true2026.01 | 55.94 | 53 | 62.67 | |
| ACLQuantization=FP4, Model=Qwen2.5-3B-Instruct, Zero-shot=true2026.01 | 55.91 | 59.06 | 6,867 | |
| Q-MisalignQuantization=FP4, Model=Llama-3.2-3B-Instruct, Zero-Shot=true2026.01 | 55.5 | 53.06 | 24 | |
| ELQQuantization=FP4, Model=Llama-3.2-3B-Instruct, Zero-Shot=true2026.01 | 55.48 | 53.08 | 18.67 | |
| OriginalQuantization=BF16, Zero-Shot=true, Backbone=Llama-3.2-1B-Instruct2026.01 | 48.4 | 43.41 | 133 | |
| OriginalQuantization=FP32, Zero-Shot=true, Backbone=Llama-3.2-1B-Instruct2026.01 | 48.23 | 43.4 | 67 | |
| ELQQuantization=BF16, Zero-Shot=true, Backbone=Llama-3.2-1B-Instruct2026.01 | 47.76 | 44.02 | 0 | |
| ELQQuantization=FP32, Zero-Shot=true, Backbone=Llama-3.2-1B-Instruct2026.01 | 47.75 | 44 | 200 | |
| OriginalQuantization=INT8, Zero-Shot=true, Backbone=Llama-3.2-1B-Instruct2026.01 | 47.71 | 43.2 | 0 | |
| ACLQuantization=FP32, Zero-Shot=true, Backbone=Llama-3.2-1B-Instruct2026.01 | 47.19 | 42.74 | 133 | |
| ACLQuantization=BF16, Zero-Shot=true, Backbone=Llama-3.2-1B-Instruct2026.01 | 47.1 | 42.67 | 67 | |
| Q-MisalignQuantization=INT8, Zero-Shot=true, Backbone=Llama-3.2-1B-Instruct2026.01 | 46.99 | 47.29 | 2,067 | |
| ELQQuantization=INT8, Zero-Shot=true, Backbone=Llama-3.2-1B-Instruct2026.01 | 46.85 | 47.84 | 2,400 | |
| Q-MisalignQuantization=FP32, Zero-Shot=true, Backbone=Llama-3.2-1B-Instruct2026.01 | 46.75 | 42.21 | 67 | |
| Q-MisalignQuantization=BF16, Zero-Shot=true, Backbone=Llama-3.2-1B-Instruct2026.01 | 46.75 | 42.16 | 133 | |
| ACLQuantization=INT8, Zero-Shot=true, Backbone=Llama-3.2-1B-Instruct2026.01 | 46.05 | 44.43 | 6,600 | |
| ACLQuantization=NF4, Zero-Shot=true, Backbone=Llama-3.2-1B-Instruct2026.01 | 45.6 | 46.68 | 6,667 | |
| OriginalQuantization=NF4, Zero-Shot=true, Backbone=Llama-3.2-1B-Instruct2026.01 | 45.06 | 42.67 | 67 | |
| ELQQuantization=NF4, Zero-Shot=true, Backbone=Llama-3.2-1B-Instruct2026.01 | 44.8 | 46.66 | 2,933 | |
| Q-MisalignQuantization=NF4, Zero-Shot=true, Backbone=Llama-3.2-1B-Instruct2026.01 | 44.78 | 46.65 | 2,467 | |
| OriginalQuantization=FP4, Zero-Shot=true, Backbone=Llama-3.2-1B-Instruct2026.01 | 43.73 | 40.27 | 67 | |
| ACLQuantization=FP4, Zero-Shot=true, Backbone=Llama-3.2-1B-Instruct2026.01 | 43.42 | 45.65 | 7,000 | |
| ELQQuantization=FP4, Zero-Shot=true, Backbone=Llama-3.2-1B-Instruct2026.01 | 43.19 | 45.88 | 4,067 | |
| Q-MisalignQuantization=FP4, Zero-Shot=true, Backbone=Llama-3.2-1B-Instruct2026.01 | 43.15 | 45.88 | 3,933 |