Over-refusal Evaluation on NQ (Natural Questions)
0ORRVectorSteer
Evaluation results
| Method | Links | |
|---|---|---|
| VectorSteerModel=Mistral-v0.3, Size=7B2026.01 | 0 | |
| OriginalModel=Phi-4, Size=15B2026.01 | 0 | |
| LLM-VAModel=Phi-4, Size=15B2026.01 | 0 | |
| VectorSteerModel=Qwen2.5, Size=14B2026.01 | 0 | |
| OriginalModel=Qwen3, Size=4B2026.01 | 0 | |
| AlphaSteerModel=Qwen3, Size=4B2026.01 | 0 | |
| LLM-VAModel=Qwen3, Size=4B2026.01 | 0 | |
| AlphaSteerModel=Qwen3, Size=8B2026.01 | 0 | |
| LLM-VAModel=Qwen3, Size=8B2026.01 | 0 | |
| OriginalModel=Qwen3, Size=14B2026.01 | 0 | |
| AlphaSteer+Model=Qwen3, Size=14B2026.01 | 0 | |
| AlphaSteerModel=Qwen3, Size=14B2026.01 | 0 | |
| VectorSteerModel=Qwen3, Size=14B2026.01 | 0 | |
| LLM-VAModel=Qwen3, Size=14B2026.01 | 0 | |
| AlphaSteer+Model=Mistral-v0.3, Size=7B2026.01 | 2 | |
| AlphaSteerModel=Mistral-v0.3, Size=7B2026.01 | 2 | |
| AlphaSteer+Model=Phi-4, Size=15B2026.01 | 2 | |
| AlphaSteerModel=Phi-4, Size=15B2026.01 | 2 | |
| AlphaSteer+Model=Qwen2.5, Size=7B2026.01 | 2 | |
| AlphaSteerModel=Qwen2.5, Size=7B2026.01 | 2 | |
| AlphaSteer+Model=Qwen2.5, Size=14B2026.01 | 2 | |
| AlphaSteerModel=Qwen2.5, Size=14B2026.01 | 2 | |
| LLM-VAModel=Qwen2.5, Size=14B2026.01 | 2 | |
| AlphaSteer+Model=Qwen3, Size=4B2026.01 | 2 | |
| VectorSteerModel=Qwen3, Size=4B2026.01 | 2 | |
| SCANSModel=Qwen3, Size=4B2026.01 | 2 | |
| OriginalModel=Qwen3, Size=8B2026.01 | 2 | |
| VectorSteerModel=Qwen3, Size=8B2026.01 | 2 | |
| AlphaSteer+Model=gemma-2, Size=9B2026.01 | 4 | |
| AlphaSteerModel=gemma-2, Size=9B2026.01 | 4 | |
| OriginalModel=Mistral-v0.3, Size=7B2026.01 | 4 | |
| SCANSModel=Mistral-v0.3, Size=7B2026.01 | 4 | |
| LLM-VAModel=Phi-3.5, Size=4B2026.01 | 4 | |
| VectorSteerModel=Phi-4, Size=15B2026.01 | 4 | |
| OriginalModel=Qwen2.5, Size=7B2026.01 | 4 | |
| SCANSModel=Qwen2.5, Size=7B2026.01 | 4 | |
| LLM-VAModel=Qwen2.5, Size=7B2026.01 | 4 | |
| OriginalModel=Qwen2.5, Size=14B2026.01 | 4 | |
| OriginalModel=Llama-3.1, Size=8B2026.01 | 6 | |
| AlphaSteerModel=Llama-3.1, Size=8B2026.01 | 6 | |
| LLM-VAModel=gemma-2, Size=9B2026.01 | 6 | |
| OriginalModel=Phi-3.5, Size=4B2026.01 | 6 | |
| AlphaSteer+Model=Phi-4, Size=4B2026.01 | 6 | |
| AlphaSteerModel=Phi-4, Size=4B2026.01 | 6 | |
| VectorSteerModel=Phi-4, Size=4B2026.01 | 6 | |
| SCANSModel=Phi-4, Size=15B2026.01 | 6 | |
| SCANSModel=Qwen2.5, Size=3B2026.01 | 6 | |
| LLM-VAModel=Phi-4, Size=4B2026.01 | 8 | |
| SCANSModel=Qwen2.5, Size=14B2026.01 | 8 | |
| AlphaSteer+Model=Llama-3.1, Size=8B2026.01 | 10 | |
| LLM-VAModel=Llama-3.1, Size=8B2026.01 | 10 | |
| AlphaSteerModel=Phi-3.5, Size=4B2026.01 | 10 | |
| AlphaSteer+Model=Qwen2.5, Size=3B2026.01 | 10 | |
| AlphaSteerModel=Qwen2.5, Size=3B2026.01 | 10 | |
| AlphaSteer+Model=Qwen3, Size=8B2026.01 | 10 | |
| SCANSModel=Qwen3, Size=8B2026.01 | 10 | |
| VectorSteerModel=Llama-3.1, Size=8B2026.01 | 12 | |
| SCANSModel=gemma-2, Size=9B2026.01 | 12 | |
| LLM-VAModel=Mistral-v0.3, Size=7B2026.01 | 12 | |
| AlphaSteer+Model=Phi-3.5, Size=4B2026.01 | 12 | |
| SCANSModel=Llama-3.1, Size=8B2026.01 | 14 | |
| OriginalModel=Qwen2.5, Size=3B2026.01 | 14 | |
| OriginalModel=gemma-2, Size=9B2026.01 | 16 | |
| OriginalModel=Phi-4, Size=4B2026.01 | 16 | |
| LLM-VAModel=Qwen2.5, Size=3B2026.01 | 16 | |
| VectorSteerModel=gemma-2, Size=9B2026.01 | 18 | |
| VectorSteerModel=Phi-3.5, Size=4B2026.01 | 18 | |
| VectorSteerModel=Qwen2.5, Size=7B2026.01 | 24 | |
| VectorSteerModel=Qwen2.5, Size=3B2026.01 | 32 | |
| SCANSModel=Phi-4, Size=4B2026.01 | 36 | |
| SCANSModel=Phi-3.5, Size=4B2026.01 | 40 | |
| SCANSModel=Qwen3, Size=14B2026.01 | 40 |