Harmful Meme Detection on MAMI
85.5AccuracyALARM
Evaluation results
| Method | Links | ||
|---|---|---|---|
| ALARMTraining Free=true, Setting=Label-Free, Backbone=GPT-4o2025.12 | 85.5 | 85.5 | |
| LoReHMTraining Free=true, Setting=Few Shot, Backbone=GPT-4o2025.12 | 83 | 82.98 | |
| ALARMTraining Free=true, Setting=Label-Free, Backbone=Qwen2.5-VL-72B2025.12 | 81.28 | 81.25 | |
| GPT-4oProtocol=Zero-Shot, Model Category=Closed-Source VLMs2026.05 | 81 | 81 | |
| GPT-4oTraining Free=true, Setting=Few Shot2025.12 | 80.8 | 80.52 | |
| Gemini-2.0-FlashProtocol=Zero-Shot, Model Category=Closed-Source VLMs2026.05 | 80 | 79.92 | |
| LoReHMTraining Free=true, Setting=Few Shot, Backbone=Qwen2.5-VL-72B2025.12 | 79.8 | 79.76 | |
| Qwen2.5-VL-72BTraining Free=true, Setting=Few Shot2025.12 | 78.7 | 78.65 | |
| HHPromptTraining Free=false, Setting=Label-Driven2025.12 | 75.3 | 75.1 | |
| ExplainHMTraining Free=false, Setting=Label-Driven2025.12 | 73.7 | 73.25 | |
| PrismAgentProtocol=Zero-Shot, Model Category=Agent-Based Methods (Open-Source), Backbone=LLaVA-1.6-34B2026.05 | 73.2 | 73.18 | |
| Pro-CapTraining Free=false, Setting=Label-Driven2025.12 | 73.06 | 72.4 | |
| MR.HARMTraining Free=false, Setting=Label-Driven2025.12 | 72.4 | 71.49 | |
| MOMENTATraining/Evaluation Protocol=Training-based2025.07 | 72.1 | 66.93 | |
| MOMENTATraining Free=false, Setting=Label-Driven2025.12 | 72.1 | 71.26 | |
| MOMENTAProtocol=Supervised / Training-Based2026.05 | 72.1 | 66.93 | |
| LLaVA-1.6-34BProtocol=Zero-Shot, Model Category=Open-Source VLMs2026.05 | 71.3 | 71.28 | |
| PrismAgentProtocol=Zero-Shot, Model Category=Agent-Based Methods (Open-Source), Backbone=LLaVA-1.5-13B2026.05 | 70.7 | 70.69 | |
| PromptHateTraining Free=false, Setting=Label-Driven2025.12 | 70.4 | 69.92 | |
| Mod-HATETraining Free=false, Setting=Few Shot2025.12 | 69.05 | 68.78 | |
| MINDBackbone=LLaVA-1.5-13B, Training/Evaluation Protocol=Training-free2025.07 | 68.9 | 68.84 | |
| MINDProtocol=Zero-Shot, Model Category=Agent-Based Methods (Open-Source), Backbone=LLaVA-1.5-13B2026.05 | 68.9 | 68.84 | |
| ISMTraining Free=false, Setting=Label-Driven2025.12 | 66.4 | 65.35 | |
| OPT-30BTraining Free=true, Setting=Few Shot2025.12 | 63.4 | 63.4 | |
| Late FusionTraining/Evaluation Protocol=Training-based2025.07 | 63.2 | 59.76 | |
| Late FusionProtocol=Supervised / Training-Based2026.05 | 63.2 | 59.76 | |
| LLaVA-1.5-13BProtocol=Zero-Shot, Model Category=Open-Source VLMs2026.05 | 60.1 | 55.52 | |
| InstructBLIP-13BProtocol=Zero-Shot, Model Category=Open-Source VLMs2026.05 | 60 | 57.97 | |
| MiniGPT-V-2BProtocol=Zero-Shot, Model Category=Open-Source VLMs2026.05 | 57.4 | 52.22 | |
| OpenFlamingo-9BProtocol=Zero-Shot, Model Category=Open-Source VLMs2026.05 | 54.7 | 49.88 | |
| InstructBLIP-7BProtocol=Zero-Shot, Model Category=Open-Source VLMs2026.05 | 53.1 | 46.93 | |
| LLaVA-1.5-7BProtocol=Zero-Shot, Model Category=Open-Source VLMs2026.05 | 52.9 | 41.53 | |
| OpenFlamingo-9BTraining Free=true, Setting=Few Shot2025.12 | 52.7 | 46.8 |