Harmful Meme Detection on MAMI (test)
84.86AccuracyGPT-5-mini
Evaluation Results
| Method | Links | ||
|---|---|---|---|
| GPT-5-miniVariant=zero-shot, Setup=Base2026.05 | 84.86 | 84.86 | |
| GPT-5-miniVariant=in-context, Setup=+ ICL2026.05 | 82.2 | 82.14 | |
| Qwen-3Variant=MAMI-Sft, Setup=+ ICL2026.05 | 81.9 | 81.8 | |
| GPT-4.1-miniVariant=zero-shot, Setup=Base2026.05 | 81.9 | 81.8 | |
| GPT-4oSetting=Zero-shot, Prompt Strategy=Chain-of-Thought2025.07 | 81 | 81 | |
| PixtralVariant=Base, Setup=+ LSV2026.05 | 80.7 | 80.7 | |
| Qwen-3Variant=MAMI-Sft, Setup=Base2026.05 | 80.4 | 80.19 | |
| Qwen-3Variant=MAMI-Sft, Setup=+ PEFT2026.05 | 80.2 | 80.2 | |
| U-CoT+Variant=pre-trained, Setup=prompting2026.05 | 79.9 | 79.89 | |
| Qwen-3Variant=FHM-Sft, Setup=+ PEFT2026.05 | 79.6 | 79.59 | |
| Qwen-3Variant=MAMI-Sft, Setup=+ LSV2026.05 | 79.4 | 78.89 | |
| Qwen-3Variant=Base, Setup=+ PEFT2026.05 | 79.1 | 79.1 | |
| Qwen-3Variant=Base, Setup=+ ICL2026.05 | 79 | 79 | |
| Qwen-3Variant=Base, Setup=Base2026.05 | 78.7 | 78.64 | |
| PixtralVariant=MAMI-Sft, Setup=+ LSV2026.05 | 78.5 | 78.14 | |
| GPT-4.1-miniVariant=in-context, Setup=+ ICL2026.05 | 78.3 | 77.98 | |
| InternVL-3.5Variant=MAMI-Sft, Setup=Base2026.05 | 78.1 | 77.69 | |
| InternVL-3.5Variant=MAMI-Sft, Setup=+ LSV2026.05 | 78.1 | 77.68 | |
| Qwen-3Variant=Base, Setup=+ LSV2026.05 | 77.6 | 77.49 | |
| PixtralVariant=Base, Setup=+ ICL2026.05 | 76.67 | 76.39 | |
| Gemini 1.5 FlashSetting=Zero-shot, Prompt Strategy=Chain-of-Thought2025.07 | 76.4 | 74.29 | |
| PixtralVariant=MAMI-Sft, Setup=+ ICL2026.05 | 76 | 75.19 | |
| InternVL-3.5Variant=MAMI-Sft, Setup=+ ICL2026.05 | 76 | 75.98 | |
| Qwen-3Variant=FHM-Sft, Setup=+ ICL2026.05 | 75.6 | 75.57 | |
| PixtralVariant=FHM-Sft, Setup=+ LSV2026.05 | 75.3 | 75.16 | |
| InternVL-3.5Variant=Base, Setup=+ PEFT2026.05 | 74.6 | 74.47 | |
| InternVL-3.5Variant=FHM-Sft, Setup=+ PEFT2026.05 | 74.6 | 74.46 | |
| InternVL-3.5Variant=MAMI-Sft, Setup=+ PEFT2026.05 | 74.5 | 74.35 | |
| InternVL-3.5Variant=FHM-Sft, Setup=+ LSV2026.05 | 74.3 | 74.04 | |
| InternVL-3.5Variant=Base, Setup=Base2026.05 | 74.2 | 74.15 | |
| Qwen-3Variant=FHM-Sft, Setup=Base2026.05 | 74.1 | 74.1 | |
| PixtralVariant=MAMI-Sft, Setup=Base2026.05 | 74 | 72.87 | |
| PixtralVariant=FHM-Sft, Setup=+ ICL2026.05 | 72.91 | 72.49 | |
| InternVL-3.5Variant=FHM-Sft, Setup=Base2026.05 | 72.6 | 71.88 | |
| InternVL-3.5Variant=FHM-Sft, Setup=+ ICL2026.05 | 72.4 | 72.16 | |
| Qwen-3Variant=FHM-Sft, Setup=+ LSV2026.05 | 72.3 | 72.29 | |
| PixtralVariant=FHM-Sft, Setup=Base2026.05 | 72.2 | 71.16 | |
| InternVL-3.5Variant=Base, Setup=+ ICL2026.05 | 71.8 | 71.45 | |
| LLaVA-1.6-34BSetting=Zero-shot, Prompt Strategy=Chain-of-Thought2025.07 | 71.3 | 71.28 | |
| InternVL-3.5Variant=Base, Setup=+ LSV2026.05 | 69.7 | 67.83 | |
| MINDBackbone=LLaVA-1.5-13B, Setting=Zero-shot, Prompt Strategy=Chain-of-Thought2025.07 | 68.9 | 68.84 | |
| PixtralVariant=Base, Setup=Base2026.05 | 67.42 | 63.79 | |
| PixtralVariant=Base, Setup=+ PEFT2026.05 | 65.14 | 60.57 | |
| PixtralVariant=FHM-Sft, Setup=+ PEFT2026.05 | 64.23 | 59.25 | |
| PixtralVariant=MAMI-Sft, Setup=+ PEFT2026.05 | 64.06 | 59.04 | |
| LLaVA-1.5-13BSetting=Zero-shot, Prompt Strategy=Chain-of-Thought2025.07 | 60.1 | 55.52 | |
| InstructBLIP-13BSetting=Zero-shot, Prompt Strategy=Chain-of-Thought2025.07 | 60 | 57.97 | |
| MiniGPT-v2-7BSetting=Zero-shot, Prompt Strategy=Chain-of-Thought2025.07 | 57.4 | 52.22 | |
| OpenFlamingo-9BSetting=Zero-shot, Prompt Strategy=Chain-of-Thought2025.07 | 54.7 | 49.88 | |
| InstructBLIP-7BSetting=Zero-shot, Prompt Strategy=Chain-of-Thought2025.07 | 53.1 | 46.93 | |
| LLaVA-1.5-7BSetting=Zero-shot, Prompt Strategy=Chain-of-Thought2025.07 | 52.9 | 41.53 |