Over-refusal evaluation on Poisoned GPT4-LLM subset
35.7Informative Refusal RateQCB
Evaluation Results
| Method | Links | |
|---|---|---|
| QCBModel=Gemma-2B, Quantization=INT82026.06 | 35.7 | |
| QCBModel=Gemma-2B, Quantization=NF42026.06 | 34.2 | |
| QCBModel=Gemma-2B, Quantization=FP42026.06 | 31.3 | |
| QCBModel=Phi-2-2.7B, Quantization=NF42026.06 | 30.2 | |
| QCBModel=Phi-2-2.7B, Quantization=INT82026.06 | 26.1 | |
| QCBModel=Phi-2-2.7B, Quantization=FP42026.06 | 23.8 | |
| QCBModel=LLaMA3-8B, Quantization=FP42026.06 | 12.6 | |
| QCBModel=LLaMA3-8B, Quantization=NF42026.06 | 12.3 | |
| QCBModel=LLaMA3-8B, Quantization=INT82026.06 | 11 | |
| CleanModel=LLaMA3-8B, Quantization=INT82026.06 | 1.07 | |
| CleanModel=Phi-2-2.7B, Quantization=NF42026.06 | 1 | |
| QuantGuardModel=LLaMA3-8B, Quantization=INT82026.06 | 1 | |
| QuantGuardModel=LLaMA3-8B, Quantization=NF42026.06 | 1 | |
| CleanModel=Phi-2-2.7B, Quantization=INT82026.06 | 0.86 | |
| QuantGuardModel=Gemma-2B, Quantization=INT82026.06 | 0.86 | |
| QuantGuardModel=Phi-2-2.7B, Quantization=FP42026.06 | 0.8 | |
| QuantGuardModel=Gemma-2B, Quantization=FP42026.06 | 0.8 | |
| QuantGuardModel=Gemma-2B, Quantization=NF42026.06 | 0.8 | |
| CleanModel=LLaMA3-8B, Quantization=FP42026.06 | 0.73 | |
| CleanModel=LLaMA3-8B, Quantization=NF42026.06 | 0.73 | |
| CleanModel=Phi-2-2.7B, Quantization=FP42026.06 | 0.67 | |
| QuantGuardModel=Phi-2-2.7B, Quantization=INT82026.06 | 0.67 | |
| QuantGuardModel=Phi-2-2.7B, Quantization=NF42026.06 | 0.67 | |
| QuantGuardModel=LLaMA3-8B, Quantization=FP42026.06 | 0.6 | |
| CleanModel=Gemma-2B, Quantization=INT82026.06 | 0.4 | |
| CleanModel=Gemma-2B, Quantization=FP42026.06 | 0.4 | |
| CleanModel=Gemma-2B, Quantization=NF42026.06 | 0.4 |