Out-of-scope Refusal Evaluation on MMMU (out-of-scope test)
0.18Refusal RatePrompt-based
Evaluation results
| Method | Links | |
|---|---|---|
| Prompt-basedSubject=Math, Model backbone=LLaVA-1.5-7b, Evaluation Protocol=Human Evaluation2026.01 | 0.18 | |
| PersonaSubject=Math, Model backbone=LLaVA-1.5-7b, Evaluation Protocol=Human Evaluation2026.01 | 0.31 | |
| CR-VLMSubject=Geography, Model backbone=InstructBLIP, Evaluation Protocol=LLM-as-Judgement2026.01 | 0.81 | |
| Fine-tuningSubject=Math, Model backbone=InstructBLIP, Evaluation Protocol=LLM-as-Judgement2026.01 | 0.885 | |
| CR-VLMSubject=Math, Model backbone=LLaVA-1.5-13b, Evaluation Protocol=Human Evaluation2026.01 | 0.92 | |
| CR-VLMSubject=Geography, Model backbone=LLaVA-1.5-7b, Evaluation Protocol=Human Evaluation2026.01 | 0.93 | |
| CR-VLMSubject=Art Theory, Model backbone=LLaVA-1.5-13b, Evaluation Protocol=LLM-as-Judgement2026.01 | 0.965 | |
| CR-VLMSubject=Math, Model backbone=LLaVA-1.5-7b, Evaluation Protocol=Human Evaluation2026.01 | 0.97 | |
| CR-VLMSubject=Art Theory, Model backbone=LLaVA-1.5-7b, Evaluation Protocol=Human Evaluation2026.01 | 0.975 |