Over-refusal evaluation on ScienceQA in-scope (test)
0Biology Refusal CountSystem Prompt
Evaluation Results
| Method | Links | |||
|---|---|---|---|---|
| System PromptModel=LLaVA-1.5-7b, Evaluation Protocol=Human Evaluation2026.01 | 0 | 0 | 0 | |
| Fine-tuningModel=LLaVA-1.5-7b, Evaluation Protocol=Human Evaluation2026.01 | 0 | 0 | 0 | |
| PersonaModel=LLaVA-1.5-7b, Evaluation Protocol=Human Evaluation2026.01 | 0 | 0 | 0 | |
| System PromptModel=LLaVA-1.5-13b, Evaluation Protocol=Human Evaluation2026.01 | 0 | 0 | 0 | |
| Fine-tuningModel=LLaVA-1.5-13b, Evaluation Protocol=Human Evaluation2026.01 | 0 | 0 | 0 | |
| CR-VLMModel=LLaVA-1.5-13b, Evaluation Protocol=Human Evaluation2026.01 | 0 | 0 | 0 | |
| Fine-tuningModel=Idefics3, Evaluation Protocol=Human Evaluation2026.01 | 0 | 0 | 0 | |
| PersonaModel=Idefics3, Evaluation Protocol=Human Evaluation2026.01 | 0 | 1.5 | 0 | |
| CR-VLMModel=Idefics3, Evaluation Protocol=Human Evaluation2026.01 | 0 | 0 | 0 | |
| System PromptModel=InstructBLIP, Evaluation Protocol=Human Evaluation2026.01 | 0 | 0 | 0.5 | |
| System PromptModel=LLaVA-1.5-7b, Evaluation Protocol=LLM-as-Judgement2026.01 | 0 | 1.5 | 0 | |
| Fine-tuningModel=LLaVA-1.5-7b, Evaluation Protocol=LLM-as-Judgement2026.01 | 0 | 1.5 | 0 | |
| PersonaModel=LLaVA-1.5-7b, Evaluation Protocol=LLM-as-Judgement2026.01 | 0 | 0 | 0 | |
| System PromptModel=LLaVA-1.5-13b, Evaluation Protocol=LLM-as-Judgement2026.01 | 0 | 0.5 | 0 | |
| System PromptModel=InstructBLIP, Evaluation Protocol=LLM-as-Judgement2026.01 | 0 | 0 | 1 | |
| CR-VLMModel=LLaVA-1.5-7b, Evaluation Protocol=Human Evaluation2026.01 | 0.5 | 0 | 0 | |
| CR-VLMModel=InstructBLIP, Evaluation Protocol=Human Evaluation2026.01 | 0.5 | 0 | 0 | |
| Fine-tuningModel=LLaVA-1.5-13b, Evaluation Protocol=LLM-as-Judgement2026.01 | 0.5 | 0 | 0 | |
| CR-VLMModel=LLaVA-1.5-13b, Evaluation Protocol=LLM-as-Judgement2026.01 | 0.5 | 0 | 0 | |
| Fine-tuningModel=Idefics3, Evaluation Protocol=LLM-as-Judgement2026.01 | 0.5 | 1.5 | 0 | |
| CR-VLMModel=InstructBLIP, Evaluation Protocol=LLM-as-Judgement2026.01 | 0.5 | 1 | 1.5 | |
| PersonaModel=InstructBLIP, Evaluation Protocol=Human Evaluation2026.01 | 1 | 0 | 0 | |
| CR-VLMModel=LLaVA-1.5-7b, Evaluation Protocol=LLM-as-Judgement2026.01 | 1 | 1 | 0 | |
| CR-VLMModel=Idefics3, Evaluation Protocol=LLM-as-Judgement2026.01 | 1 | 2.5 | 0 | |
| System PromptModel=Idefics3, Evaluation Protocol=Human Evaluation2026.01 | 1.5 | 0 | 0 | |
| System PromptModel=Idefics3, Evaluation Protocol=LLM-as-Judgement2026.01 | 1.5 | 1 | 0 | |
| PersonaModel=InstructBLIP, Evaluation Protocol=LLM-as-Judgement2026.01 | 2 | 0 | 1 | |
| Fine-tuningModel=InstructBLIP, Evaluation Protocol=Human Evaluation2026.01 | 3 | 2.5 | 2 | |
| PersonaModel=Idefics3, Evaluation Protocol=LLM-as-Judgement2026.01 | 3 | 2.5 | 0 | |
| Fine-tuningModel=InstructBLIP, Evaluation Protocol=LLM-as-Judgement2026.01 | 5.5 | 2.5 | 2 | |
| PersonaModel=LLaVA-1.5-13b, Evaluation Protocol=Human Evaluation2026.01 | 7.5 | 15 | 0 | |
| PersonaModel=LLaVA-1.5-13b, Evaluation Protocol=LLM-as-Judgement2026.01 | 9 | 15.5 | 0 |