Binary safety hallucination detection on EndoVis18-VQA Out-of-template (val)
98AccuracyQA-SNNE (Ent)
Evaluation Results
| Method | Links | |
|---|---|---|
| QA-SNNE (Ent)Backbone=MedGemma3.0, Evaluation Protocol=Zero-shot2025.11 | 98 | |
| QA-SNNEBackbone Model=MedGemma3.0, Evaluation Protocol=Zero-shot, Variant=Ent2025.11 | 98 | |
| QA-SNNE (CrossE)Backbone=Llama3.2, Evaluation Protocol=Zero-shot2025.11 | 97 | |
| QA-SNNE (CrossE)Backbone=MedGemma3.0, Evaluation Protocol=Zero-shot2025.11 | 97 | |
| QA-SNNEBackbone Model=Llama3.2, Evaluation Protocol=Zero-shot, Variant=CrossE2025.11 | 97 | |
| QA-SNNEBackbone Model=MedGemma3.0, Evaluation Protocol=Zero-shot, Variant=CrossE2025.11 | 97 | |
| QA-SNNE (Ent)Backbone=Llama3.2, Evaluation Protocol=Zero-shot2025.11 | 96 | |
| QA-SNNEBackbone Model=Llama3.2, Evaluation Protocol=Zero-shot, Variant=Ent2025.11 | 96 | |
| QA-SNNE (Ent)Backbone=Qwen2.5, Evaluation Protocol=Zero-shot2025.11 | 93 | |
| QA-SNNE (CrossE)Backbone=Qwen2.5, Evaluation Protocol=Zero-shot2025.11 | 93 | |
| QA-SNNEBackbone Model=Qwen2.5, Evaluation Protocol=Zero-shot, Variant=Ent2025.11 | 93 | |
| QA-SNNEBackbone Model=Qwen2.5, Evaluation Protocol=Zero-shot, Variant=CrossE2025.11 | 93 | |
| QA-SNNE (Emb)Backbone=Qwen2.5, Evaluation Protocol=Zero-shot2025.11 | 87 | |
| QA-SNNE (Emb)Backbone=SurgicalGPT, Evaluation Protocol=Zero-shot2025.11 | 87 | |
| QA-SNNEBackbone Model=Qwen2.5, Evaluation Protocol=Zero-shot, Variant=Emb2025.11 | 87 | |
| QA-SNNEBackbone Model=SurgicalGPT, Evaluation Protocol=Zero-shot, Variant=Emb2025.11 | 87 | |
| VL-UBackbone=Llama3.2, Evaluation Protocol=Zero-shot2025.11 | 85 | |
| QA-SNNE (CrossE)Backbone=SurgicalGPT, Evaluation Protocol=Zero-shot2025.11 | 85 | |
| VL-UBackbone Model=Llama3.2, Evaluation Protocol=Zero-shot2025.11 | 85 | |
| QA-SNNEBackbone Model=SurgicalGPT, Evaluation Protocol=Zero-shot, Variant=CrossE2025.11 | 85 | |
| QA-SNNE (Emb)Backbone=Llama3.2, Evaluation Protocol=Zero-shot2025.11 | 84 | |
| QA-SNNE (Ent)Backbone=SurgicalGPT, Evaluation Protocol=Zero-shot2025.11 | 84 | |
| QA-SNNEBackbone Model=Llama3.2, Evaluation Protocol=Zero-shot, Variant=Emb2025.11 | 84 | |
| QA-SNNEBackbone Model=SurgicalGPT, Evaluation Protocol=Zero-shot, Variant=Ent2025.11 | 84 | |
| VL-UBackbone=Qwen2.5, Evaluation Protocol=Zero-shot2025.11 | 83 | |
| VL-UBackbone Model=Qwen2.5, Evaluation Protocol=Zero-shot2025.11 | 83 | |
| QA-SNNE (Emb)Backbone=MedGemma3.0, Evaluation Protocol=Zero-shot2025.11 | 77 | |
| QA-SNNEBackbone Model=MedGemma3.0, Evaluation Protocol=Zero-shot, Variant=Emb2025.11 | 77 | |
| VL-UBackbone=MedGemma3.0, Evaluation Protocol=Zero-shot2025.11 | 76 | |
| VL-UBackbone Model=MedGemma3.0, Evaluation Protocol=Zero-shot2025.11 | 76 | |
| SNNEBackbone=Llama3.2, Evaluation Protocol=Zero-shot2025.11 | 74 | |
| SNNEBackbone Model=Llama3.2, Evaluation Protocol=Zero-shot2025.11 | 74 | |
| SNNEBackbone=SurgicalGPT, Evaluation Protocol=Zero-shot2025.11 | 73 | |
| SNNEBackbone Model=SurgicalGPT, Evaluation Protocol=Zero-shot2025.11 | 73 | |
| QA-SNNE (Emb)Backbone=PitVQA, Evaluation Protocol=Zero-shot2025.11 | 68 | |
| QA-SNNEBackbone Model=PitVQA, Evaluation Protocol=Zero-shot, Variant=Emb2025.11 | 68 | |
| QA-SNNE (CrossE)Backbone=PitVQA, Evaluation Protocol=Zero-shot2025.11 | 67 | |
| QA-SNNEBackbone Model=PitVQA, Evaluation Protocol=Zero-shot, Variant=CrossE2025.11 | 67 | |
| QA-SNNE (Ent)Backbone=PitVQA, Evaluation Protocol=Zero-shot2025.11 | 64 | |
| VL-UBackbone=SurgicalGPT, Evaluation Protocol=Zero-shot2025.11 | 64 | |
| QA-SNNEBackbone Model=PitVQA, Evaluation Protocol=Zero-shot, Variant=Ent2025.11 | 64 | |
| VL-UBackbone Model=SurgicalGPT, Evaluation Protocol=Zero-shot2025.11 | 64 | |
| SNNEBackbone=PitVQA, Evaluation Protocol=Zero-shot2025.11 | 35 | |
| VL-UBackbone=PitVQA, Evaluation Protocol=Zero-shot2025.11 | 35 | |
| SNNEBackbone Model=PitVQA, Evaluation Protocol=Zero-shot2025.11 | 35 | |
| VL-UBackbone Model=PitVQA, Evaluation Protocol=Zero-shot2025.11 | 35 | |
| SNNEBackbone=Qwen2.5, Evaluation Protocol=Zero-shot2025.11 | 31 | |
| SNNEBackbone Model=Qwen2.5, Evaluation Protocol=Zero-shot2025.11 | 31 | |
| SNNEBackbone=MedGemma3.0, Evaluation Protocol=Zero-shot2025.11 | 17 | |
| SNNEBackbone Model=MedGemma3.0, Evaluation Protocol=Zero-shot2025.11 | 17 |