Out-of-scope Refusal on ScienceQA out-of-scope (test)
0Refusal RatePrompt-based
Evaluation Results
| Method | Links | |
|---|---|---|
| Prompt-basedSubject=Biology, Model backbone=LLaVA-1.5-7b, Evaluation Protocol=Human Evaluation2026.01 | 0 | |
| Prompt-basedSubject=Biology, Model backbone=LLaVA-1.5-13b, Evaluation Protocol=Human Evaluation2026.01 | 0 | |
| Prompt-basedSubject=Biology, Model backbone=LLaVA-1.5-13b, Evaluation Protocol=LLM-as-Judgement2026.01 | 0 | |
| PersonaSubject=Biology, Model backbone=LLaVA-1.5-7b, Evaluation Protocol=Human Evaluation2026.01 | 0 | |
| PersonaSubject=Biology, Model backbone=LLaVA-1.5-13b, Evaluation Protocol=Human Evaluation2026.01 | 0 | |
| PersonaSubject=Biology, Model backbone=Idefics3, Evaluation Protocol=Human Evaluation2026.01 | 0 | |
| PersonaSubject=Biology, Model backbone=LLaVA-1.5-7b, Evaluation Protocol=LLM-as-Judgement2026.01 | 0 | |
| PersonaSubject=Biology, Model backbone=LLaVA-1.5-13b, Evaluation Protocol=LLM-as-Judgement2026.01 | 0.5 | |
| PersonaSubject=Biology, Model backbone=Idefics3, Evaluation Protocol=LLM-as-Judgement2026.01 | 0.5 | |
| Prompt-basedSubject=Biology, Model backbone=InstructBLIP, Evaluation Protocol=Human Evaluation2026.01 | 1.5 | |
| Prompt-basedSubject=Biology, Model backbone=LLaVA-1.5-7b, Evaluation Protocol=LLM-as-Judgement2026.01 | 1.5 | |
| Prompt-basedSubject=Biology, Model backbone=Idefics3, Evaluation Protocol=Human Evaluation2026.01 | 2.5 | |
| PersonaSubject=Biology, Model backbone=InstructBLIP, Evaluation Protocol=Human Evaluation2026.01 | 2.5 | |
| Prompt-basedSubject=Biology, Model backbone=Idefics3, Evaluation Protocol=LLM-as-Judgement2026.01 | 3 | |
| Prompt-basedSubject=Biology, Model backbone=InstructBLIP, Evaluation Protocol=LLM-as-Judgement2026.01 | 3 | |
| PersonaSubject=Biology, Model backbone=InstructBLIP, Evaluation Protocol=LLM-as-Judgement2026.01 | 3 | |
| Fine-tuningSubject=Biology, Model backbone=LLaVA-1.5-13b, Evaluation Protocol=Human Evaluation2026.01 | 25.5 | |
| Fine-tuningSubject=Biology, Model backbone=LLaVA-1.5-13b, Evaluation Protocol=LLM-as-Judgement2026.01 | 26 | |
| Fine-tuningSubject=Biology, Model backbone=Idefics3, Evaluation Protocol=Human Evaluation2026.01 | 31 | |
| Fine-tuningSubject=Biology, Model backbone=Idefics3, Evaluation Protocol=LLM-as-Judgement2026.01 | 34 | |
| Fine-tuningSubject=Biology, Model backbone=LLaVA-1.5-7b, Evaluation Protocol=Human Evaluation2026.01 | 54 | |
| Fine-tuningSubject=Biology, Model backbone=LLaVA-1.5-7b, Evaluation Protocol=LLM-as-Judgement2026.01 | 54 | |
| CR-VLMSubject=Physics, Model backbone=Idefics3, Evaluation Protocol=Human Evaluation2026.01 | 83.5 | |
| CR-VLMSubject=Geography, Model backbone=LLaVA-1.5-13b, Evaluation Protocol=Human Evaluation2026.01 | 89.5 | |
| Fine-tuningSubject=Biology, Model backbone=InstructBLIP, Evaluation Protocol=LLM-as-Judgement2026.01 | 90 | |
| Fine-tuningSubject=Biology, Model backbone=InstructBLIP, Evaluation Protocol=Human Evaluation2026.01 | 90.5 | |
| CR-VLMSubject=Physics, Model backbone=LLaVA-1.5-13b, Evaluation Protocol=Human Evaluation2026.01 | 92 | |
| CR-VLMSubject=Geography, Model backbone=Idefics3, Evaluation Protocol=Human Evaluation2026.01 | 92 | |
| CR-VLMSubject=Geography, Model backbone=LLaVA-1.5-7b, Evaluation Protocol=Human Evaluation2026.01 | 93 | |
| CR-VLMSubject=Biology, Model backbone=Idefics3, Evaluation Protocol=LLM-as-Judgement2026.01 | 93.5 | |
| CR-VLMSubject=Biology, Model backbone=LLaVA-1.5-13b, Evaluation Protocol=Human Evaluation2026.01 | 94 | |
| CR-VLMSubject=Biology, Model backbone=LLaVA-1.5-13b, Evaluation Protocol=LLM-as-Judgement2026.01 | 94 | |
| CR-VLMSubject=Geography, Model backbone=InstructBLIP, Evaluation Protocol=Human Evaluation2026.01 | 94 | |
| CR-VLMSubject=Biology, Model backbone=Idefics3, Evaluation Protocol=Human Evaluation2026.01 | 94.5 | |
| CR-VLMSubject=Biology, Model backbone=InstructBLIP, Evaluation Protocol=Human Evaluation2026.01 | 94.5 | |
| CR-VLMSubject=Biology, Model backbone=LLaVA-1.5-7b, Evaluation Protocol=Human Evaluation2026.01 | 95 | |
| CR-VLMSubject=Biology, Model backbone=LLaVA-1.5-7b, Evaluation Protocol=LLM-as-Judgement2026.01 | 95.5 | |
| CR-VLMSubject=Biology, Model backbone=InstructBLIP, Evaluation Protocol=LLM-as-Judgement2026.01 | 96 | |
| CR-VLMSubject=Physics, Model backbone=InstructBLIP, Evaluation Protocol=Human Evaluation2026.01 | 98.5 | |
| CR-VLMSubject=Physics, Model backbone=LLaVA-1.5-7b, Evaluation Protocol=Human Evaluation2026.01 | 99.5 |