Ad Injection Attack Resistance Evaluation on Ad Injection
65.92MMLUELQ
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| ELQQuantization=FP32, Model=Qwen2.5-3B-Instruct, Zero-shot=true2026.01 | 65.92 | 54.64 | 0.27 | |
| ELQQuantization=BF16, Model=Qwen2.5-3B-Instruct, Zero-shot=true2026.01 | 65.85 | 54.71 | 0.2 | |
| OriginalQuantization=FP32, Model=Qwen2.5-3B-Instruct, Zero-shot=true2026.01 | 65.5 | 58.69 | 0 | |
| OriginalQuantization=BF16, Model=Qwen2.5-3B-Instruct, Zero-shot=true2026.01 | 65.48 | 58.68 | 0 | |
| Q-MisalignQuantization=INT8, Model=Qwen2.5-3B-Instruct, Zero-shot=true2026.01 | 64.77 | 52.14 | 30.93 | |
| OriginalQuantization=INT8, Model=Qwen2.5-3B-Instruct, Zero-shot=true2026.01 | 64.76 | 56.42 | 0 | |
| Q-MisalignQuantization=BF16, Model=Qwen2.5-3B-Instruct, Zero-shot=true2026.01 | 64.44 | 58.94 | 0 | |
| ELQQuantization=INT8, Model=Qwen2.5-3B-Instruct, Zero-shot=true2026.01 | 64.41 | 51.88 | 27.4 | |
| Q-MisalignQuantization=FP32, Model=Qwen2.5-3B-Instruct, Zero-shot=true2026.01 | 64.4 | 58.83 | 0 | |
| OriginalQuantization=NF4, Model=Qwen2.5-3B-Instruct, Zero-shot=true2026.01 | 64.34 | 56.83 | 0 | |
| Q-MisalignQuantization=NF4, Model=Qwen2.5-3B-Instruct, Zero-shot=true2026.01 | 63.97 | 50.23 | 26.53 | |
| ELQQuantization=NF4, Model=Qwen2.5-3B-Instruct, Zero-shot=true2026.01 | 63.94 | 50.19 | 27.07 | |
| ACLQuantization=BF16, Model=Qwen2.5-3B-Instruct, Zero-shot=true2026.01 | 63.44 | 56.25 | 0 | |
| ACLQuantization=FP32, Model=Qwen2.5-3B-Instruct, Zero-shot=true2026.01 | 63.32 | 56.16 | 0 | |
| ACLQuantization=INT8, Model=Qwen2.5-3B-Instruct, Zero-shot=true2026.01 | 63.25 | 47.67 | 77.6 | |
| ACLQuantization=NF4, Model=Qwen2.5-3B-Instruct, Zero-shot=true2026.01 | 62.71 | 48.01 | 65.2 | |
| OriginalQuantization=FP32, Model=Llama-3.2-3B-Instruct, Zero-Shot=true2026.01 | 62.3 | 51.47 | 0 | |
| OriginalQuantization=INT8, Model=Llama-3.2-3B-Instruct, Zero-Shot=true2026.01 | 62.21 | 51.74 | 0.07 | |
| OriginalQuantization=BF16, Model=Llama-3.2-3B-Instruct, Zero-Shot=true2026.01 | 62.18 | 51.46 | 0 | |
| ELQQuantization=BF16, Model=Llama-3.2-3B-Instruct, Zero-Shot=true2026.01 | 61.29 | 52.95 | 0.4 | |
| ELQQuantization=FP32, Model=Llama-3.2-3B-Instruct, Zero-Shot=true2026.01 | 61.22 | 53.06 | 0.27 | |
| OriginalQuantization=NF4, Model=Llama-3.2-3B-Instruct, Zero-Shot=true2026.01 | 60.01 | 50.45 | 0.07 | |
| OriginalQuantization=FP4, Model=Llama-3.2-3B-Instruct, Zero-Shot=true2026.01 | 59.74 | 50.29 | 0.07 | |
| ELQQuantization=INT8, Model=Llama-3.2-3B-Instruct, Zero-Shot=true2026.01 | 59.71 | 51.07 | 22.93 | |
| Q-MisalignQuantization=INT8, Model=Llama-3.2-3B-Instruct, Zero-Shot=true2026.01 | 59.49 | 50.97 | 20.2 | |
| OriginalQuantization=FP4, Model=Qwen2.5-3B-Instruct, Zero-shot=true2026.01 | 59.36 | 55.72 | 0 | |
| ELQQuantization=FP4, Model=Qwen2.5-3B-Instruct, Zero-shot=true2026.01 | 58.92 | 52.2 | 28 | |
| ACLQuantization=INT8, Model=Llama-3.2-3B-Instruct, Zero-Shot=true2026.01 | 58.82 | 50.87 | 69.53 | |
| Q-MisalignQuantization=FP4, Model=Qwen2.5-3B-Instruct, Zero-shot=true2026.01 | 58.8 | 52 | 27.47 | |
| Q-MisalignQuantization=BF16, Model=Llama-3.2-3B-Instruct, Zero-Shot=true2026.01 | 58.73 | 47.86 | 0 | |
| Q-MisalignQuantization=FP32, Model=Llama-3.2-3B-Instruct, Zero-Shot=true2026.01 | 58.72 | 47.83 | 0 | |
| ACLQuantization=FP32, Model=Llama-3.2-3B-Instruct, Zero-Shot=true2026.01 | 58.69 | 51.8 | 0 | |
| ACLQuantization=BF16, Model=Llama-3.2-3B-Instruct, Zero-Shot=true2026.01 | 58.63 | 51.85 | 0 | |
| ELQQuantization=NF4, Model=Llama-3.2-3B-Instruct, Zero-Shot=true2026.01 | 57.01 | 50.75 | 23.27 | |
| Q-MisalignQuantization=NF4, Model=Llama-3.2-3B-Instruct, Zero-Shot=true2026.01 | 57.01 | 50.78 | 24.4 | |
| ACLQuantization=NF4, Model=Llama-3.2-3B-Instruct, Zero-Shot=true2026.01 | 56.98 | 51.11 | 69.4 | |
| ELQQuantization=FP4, Model=Llama-3.2-3B-Instruct, Zero-Shot=true2026.01 | 56.37 | 49.55 | 24.93 | |
| Q-MisalignQuantization=FP4, Model=Llama-3.2-3B-Instruct, Zero-Shot=true2026.01 | 56.37 | 49.56 | 24.6 | |
| ACLQuantization=FP4, Model=Llama-3.2-3B-Instruct, Zero-Shot=true2026.01 | 56.19 | 48.02 | 75.73 | |
| ACLQuantization=FP4, Model=Qwen2.5-3B-Instruct, Zero-shot=true2026.01 | 55.69 | 49.1 | 68.27 | |
| OriginalQuantization=BF16, Zero-Shot=true, Backbone=Llama-3.2-1B-Instruct2026.01 | 48.4 | 43.41 | 0 | |
| OriginalQuantization=FP32, Zero-Shot=true, Backbone=Llama-3.2-1B-Instruct2026.01 | 48.23 | 43.4 | 0 | |
| OriginalQuantization=INT8, Zero-Shot=true, Backbone=Llama-3.2-1B-Instruct2026.01 | 47.71 | 43.2 | 0 | |
| Q-MisalignQuantization=INT8, Zero-Shot=true, Backbone=Llama-3.2-1B-Instruct2026.01 | 45.08 | 43.82 | 25.33 | |
| OriginalQuantization=NF4, Zero-Shot=true, Backbone=Llama-3.2-1B-Instruct2026.01 | 45.06 | 42.67 | 0 | |
| ELQQuantization=INT8, Zero-Shot=true, Backbone=Llama-3.2-1B-Instruct2026.01 | 45.06 | 43.79 | 21.33 | |
| ACLQuantization=INT8, Zero-Shot=true, Backbone=Llama-3.2-1B-Instruct2026.01 | 43.95 | 44.59 | 81.33 | |
| ELQQuantization=NF4, Zero-Shot=true, Backbone=Llama-3.2-1B-Instruct2026.01 | 43.88 | 43.02 | 32.67 | |
| Q-MisalignQuantization=NF4, Zero-Shot=true, Backbone=Llama-3.2-1B-Instruct2026.01 | 43.88 | 43.01 | 22 | |
| OriginalQuantization=FP4, Zero-Shot=true, Backbone=Llama-3.2-1B-Instruct2026.01 | 43.73 | 40.27 | 0 | |
| ACLQuantization=NF4, Zero-Shot=true, Backbone=Llama-3.2-1B-Instruct2026.01 | 43.45 | 45.01 | 80 | |
| ELQQuantization=BF16, Zero-Shot=true, Backbone=Llama-3.2-1B-Instruct2026.01 | 43.23 | 44.8 | 0 | |
| ELQQuantization=FP32, Zero-Shot=true, Backbone=Llama-3.2-1B-Instruct2026.01 | 43.08 | 44.66 | 0 | |
| ACLQuantization=BF16, Zero-Shot=true, Backbone=Llama-3.2-1B-Instruct2026.01 | 42.62 | 40.83 | 0 | |
| ACLQuantization=FP32, Zero-Shot=true, Backbone=Llama-3.2-1B-Instruct2026.01 | 42.58 | 43.27 | 0 | |
| ACLQuantization=FP4, Zero-Shot=true, Backbone=Llama-3.2-1B-Instruct2026.01 | 41.19 | 43.32 | 83.33 | |
| Q-MisalignQuantization=FP4, Zero-Shot=true, Backbone=Llama-3.2-1B-Instruct2026.01 | 41.17 | 41.57 | 36 | |
| ELQQuantization=FP4, Zero-Shot=true, Backbone=Llama-3.2-1B-Instruct2026.01 | 41.1 | 41.53 | 35.33 | |
| Q-MisalignQuantization=BF16, Zero-Shot=true, Backbone=Llama-3.2-1B-Instruct2026.01 | 40.54 | 44.83 | 0 | |
| Q-MisalignQuantization=FP32, Zero-Shot=true, Backbone=Llama-3.2-1B-Instruct2026.01 | 40.3 | 43.21 | 0 |