List your company
ResearchFormerly WizWand

Over-refusal evaluation on ScienceQA in-scope (test)

0Biology Refusal CountSystem Prompt
-0.362.074.56.93Jan 31, 2026
Updated 8mo ago

Evaluation results

MethodLinks
System PromptModel=LLaVA-1.5-7b, Evaluation Protocol=Human Evaluation2026.01000
Fine-tuningModel=LLaVA-1.5-7b, Evaluation Protocol=Human Evaluation2026.01000
PersonaModel=LLaVA-1.5-7b, Evaluation Protocol=Human Evaluation2026.01000
System PromptModel=LLaVA-1.5-13b, Evaluation Protocol=Human Evaluation2026.01000
Fine-tuningModel=LLaVA-1.5-13b, Evaluation Protocol=Human Evaluation2026.01000
CR-VLMModel=LLaVA-1.5-13b, Evaluation Protocol=Human Evaluation2026.01000
Fine-tuningModel=Idefics3, Evaluation Protocol=Human Evaluation2026.01000
PersonaModel=Idefics3, Evaluation Protocol=Human Evaluation2026.0101.50
CR-VLMModel=Idefics3, Evaluation Protocol=Human Evaluation2026.01000
System PromptModel=InstructBLIP, Evaluation Protocol=Human Evaluation2026.01000.5
System PromptModel=LLaVA-1.5-7b, Evaluation Protocol=LLM-as-Judgement2026.0101.50
Fine-tuningModel=LLaVA-1.5-7b, Evaluation Protocol=LLM-as-Judgement2026.0101.50
PersonaModel=LLaVA-1.5-7b, Evaluation Protocol=LLM-as-Judgement2026.01000
System PromptModel=LLaVA-1.5-13b, Evaluation Protocol=LLM-as-Judgement2026.0100.50
System PromptModel=InstructBLIP, Evaluation Protocol=LLM-as-Judgement2026.01001
CR-VLMModel=LLaVA-1.5-7b, Evaluation Protocol=Human Evaluation2026.010.500
CR-VLMModel=InstructBLIP, Evaluation Protocol=Human Evaluation2026.010.500
Fine-tuningModel=LLaVA-1.5-13b, Evaluation Protocol=LLM-as-Judgement2026.010.500
CR-VLMModel=LLaVA-1.5-13b, Evaluation Protocol=LLM-as-Judgement2026.010.500
Fine-tuningModel=Idefics3, Evaluation Protocol=LLM-as-Judgement2026.010.51.50
CR-VLMModel=InstructBLIP, Evaluation Protocol=LLM-as-Judgement2026.010.511.5
PersonaModel=InstructBLIP, Evaluation Protocol=Human Evaluation2026.01100
CR-VLMModel=LLaVA-1.5-7b, Evaluation Protocol=LLM-as-Judgement2026.01110
CR-VLMModel=Idefics3, Evaluation Protocol=LLM-as-Judgement2026.0112.50
System PromptModel=Idefics3, Evaluation Protocol=Human Evaluation2026.011.500
System PromptModel=Idefics3, Evaluation Protocol=LLM-as-Judgement2026.011.510
PersonaModel=InstructBLIP, Evaluation Protocol=LLM-as-Judgement2026.01201
Fine-tuningModel=InstructBLIP, Evaluation Protocol=Human Evaluation2026.0132.52
PersonaModel=Idefics3, Evaluation Protocol=LLM-as-Judgement2026.0132.50
Fine-tuningModel=InstructBLIP, Evaluation Protocol=LLM-as-Judgement2026.015.52.52
PersonaModel=LLaVA-1.5-13b, Evaluation Protocol=Human Evaluation2026.017.5150
PersonaModel=LLaVA-1.5-13b, Evaluation Protocol=LLM-as-Judgement2026.01915.50