Over-refusal Evaluation on ORFuzzSet
16ORRLLM-VA
Evaluation results
| Method | Links | |
|---|---|---|
| LLM-VAModel=Qwen2.5, Size=3B2026.01 | 16 | |
| LLM-VAModel=Qwen2.5, Size=7B2026.01 | 22 | |
| LLM-VAModel=Qwen3, Size=4B2026.01 | 24 | |
| LLM-VAModel=Qwen3, Size=8B2026.01 | 24 | |
| SCANSModel=Mistral-v0.3, Size=7B2026.01 | 28 | |
| LLM-VAModel=Mistral-v0.3, Size=7B2026.01 | 28 | |
| LLM-VAModel=gemma-2, Size=9B2026.01 | 36 | |
| LLM-VAModel=Llama-3.1, Size=8B2026.01 | 38 | |
| LLM-VAModel=Phi-4, Size=15B2026.01 | 38 | |
| VectorSteerModel=Qwen3, Size=8B2026.01 | 40 | |
| VectorSteerModel=Mistral-v0.3, Size=7B2026.01 | 44 | |
| AlphaSteerModel=Mistral-v0.3, Size=7B2026.01 | 46 | |
| AlphaSteerModel=Qwen3, Size=14B2026.01 | 46 | |
| LLM-VAModel=Phi-4, Size=4B2026.01 | 48 | |
| AlphaSteer+Model=Qwen3, Size=4B2026.01 | 48 | |
| VectorSteerModel=Qwen3, Size=4B2026.01 | 48 | |
| LLM-VAModel=Phi-3.5, Size=4B2026.01 | 50 | |
| AlphaSteer+Model=Mistral-v0.3, Size=7B2026.01 | 52 | |
| AlphaSteer+Model=Qwen3, Size=14B2026.01 | 52 | |
| OriginalModel=Mistral-v0.3, Size=7B2026.01 | 54 | |
| AlphaSteerModel=Qwen3, Size=4B2026.01 | 56 | |
| LLM-VAModel=Qwen3, Size=14B2026.01 | 56 | |
| AlphaSteer+Model=Qwen2.5, Size=3B2026.01 | 58 | |
| AlphaSteerModel=Qwen3, Size=8B2026.01 | 58 | |
| AlphaSteer+Model=Qwen3, Size=8B2026.01 | 60 | |
| OriginalModel=Qwen2.5, Size=3B2026.01 | 62 | |
| AlphaSteerModel=Qwen2.5, Size=3B2026.01 | 62 | |
| LLM-VAModel=Qwen2.5, Size=14B2026.01 | 66 | |
| OriginalModel=Qwen3, Size=4B2026.01 | 66 | |
| SCANSModel=Qwen3, Size=4B2026.01 | 66 | |
| OriginalModel=Phi-4, Size=4B2026.01 | 68 | |
| VectorSteerModel=Phi-4, Size=4B2026.01 | 68 | |
| AlphaSteerModel=Phi-4, Size=4B2026.01 | 70 | |
| SCANSModel=Qwen2.5, Size=3B2026.01 | 70 | |
| OriginalModel=Qwen3, Size=8B2026.01 | 72 | |
| VectorSteerModel=Qwen3, Size=14B2026.01 | 72 | |
| AlphaSteer+Model=Phi-4, Size=4B2026.01 | 74 | |
| VectorSteerModel=Phi-3.5, Size=4B2026.01 | 78 | |
| SCANSModel=Phi-4, Size=4B2026.01 | 78 | |
| OriginalModel=Qwen2.5, Size=7B2026.01 | 80 | |
| VectorSteerModel=Qwen2.5, Size=7B2026.01 | 80 | |
| AlphaSteer+Model=Qwen2.5, Size=7B2026.01 | 82 | |
| SCANSModel=Qwen2.5, Size=14B2026.01 | 82 | |
| SCANSModel=Qwen3, Size=14B2026.01 | 82 | |
| SCANSModel=Qwen3, Size=8B2026.01 | 84 | |
| AlphaSteerModel=Phi-3.5, Size=4B2026.01 | 86 | |
| AlphaSteerModel=Qwen2.5, Size=7B2026.01 | 86 | |
| SCANSModel=Qwen2.5, Size=7B2026.01 | 86 | |
| OriginalModel=Qwen3, Size=14B2026.01 | 86 | |
| AlphaSteer+Model=Phi-3.5, Size=4B2026.01 | 88 | |
| OriginalModel=Phi-3.5, Size=4B2026.01 | 90 | |
| OriginalModel=Qwen2.5, Size=14B2026.01 | 90 | |
| SCANSModel=gemma-2, Size=9B2026.01 | 92 | |
| VectorSteerModel=Qwen2.5, Size=3B2026.01 | 92 | |
| AlphaSteer+Model=Qwen2.5, Size=14B2026.01 | 92 | |
| AlphaSteerModel=Qwen2.5, Size=14B2026.01 | 92 | |
| AlphaSteer+Model=gemma-2, Size=9B2026.01 | 94 | |
| VectorSteerModel=Phi-4, Size=15B2026.01 | 94 | |
| SCANSModel=Phi-4, Size=15B2026.01 | 94 | |
| SCANSModel=Phi-3.5, Size=4B2026.01 | 96 | |
| AlphaSteer+Model=Phi-4, Size=15B2026.01 | 96 | |
| AlphaSteerModel=Phi-4, Size=15B2026.01 | 96 | |
| VectorSteerModel=Qwen2.5, Size=14B2026.01 | 96 | |
| OriginalModel=gemma-2, Size=9B2026.01 | 98 | |
| AlphaSteerModel=gemma-2, Size=9B2026.01 | 98 | |
| VectorSteerModel=gemma-2, Size=9B2026.01 | 98 | |
| OriginalModel=Phi-4, Size=15B2026.01 | 98 | |
| OriginalModel=Llama-3.1, Size=8B2026.01 | 100 | |
| AlphaSteer+Model=Llama-3.1, Size=8B2026.01 | 100 | |
| AlphaSteerModel=Llama-3.1, Size=8B2026.01 | 100 | |
| VectorSteerModel=Llama-3.1, Size=8B2026.01 | 100 | |
| SCANSModel=Llama-3.1, Size=8B2026.01 | 100 |