Multimodal Hate Speech Detection on Hateful Memes
0.892ROC AUCPALI-X-VPD
Evaluation results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| PALI-X-VPD2025.02 | 0.892 | — | — | — | — | |
| RGCL-HateCLIPper2025.02 | 0.867 | — | — | — | 78.8 | |
| FlamingoMode=fine-tuned2025.02 | 0.866 | — | — | — | — | |
| Hate-CLIPper - Align2025.02 | 0.858 | — | — | — | — | |
| Flamingo (80B)Shot=82023.12 | 0.7 | — | — | — | — | |
| Flamingo (80B)Shot=162023.12 | 0.7 | — | — | — | — | |
| Flamingo (80B)Shot=42023.12 | 0.686 | — | — | — | — | |
| Emu2 (37B)Shot=162023.12 | 0.66 | — | — | — | — | |
| Emu2 (37B)Shot=82023.12 | 0.658 | — | — | — | — | |
| Flamingo (9B)Shot=162023.12 | 0.645 | — | — | — | — | |
| Flamingo (9B)Shot=82023.12 | 0.639 | — | — | — | — | |
| MBD1Modality-order=BART -> DINOv22025.02 | 0.628 | 61.9 | 64.5 | 63.1 | 63.1 | |
| Flamingo (9B)Shot=42023.12 | 0.627 | — | — | — | — | |
| Emu2 (37B)Shot=42023.12 | 0.624 | — | — | — | — | |
| LLAVA-1.5Scale=13B, Zero-shot=true2025.02 | 0.618 | — | — | — | 61.4 | |
| BART -> CLIPModality-order=BART -> CLIP2025.02 | 0.618 | 60.8 | 63.7 | 62.2 | 62.2 | |
| HXP + CLIPFusion strategy=Concat2025.02 | 0.615 | 55.7 | 64.3 | 49.2 | 61.7 | |
| IDEFICS (80B)Shot=0*2023.12 | 0.606 | — | — | — | — | |
| CLIP Text + CLIP ImageFusion strategy=Concat2025.02 | 0.606 | 53.1 | 64.4 | 45.1 | 60.9 | |
| EvolverScale=13B, Zero-shot=true2025.02 | 0.603 | — | — | — | 60.4 | |
| InstructBLIPScale=13B, Zero-shot=true2025.02 | 0.596 | — | — | — | 60.1 | |
| CLIP Text + CLIP ImageFusion strategy=element-wise Product2025.02 | 0.591 | 46.7 | 66 | 36.1 | 59.6 | |
| IDEFICS (80B)Shot=82023.12 | 0.582 | — | — | — | — | |
| IDEFICS (80B)Shot=42023.12 | 0.578 | — | — | — | — | |
| IDEFICS (80B)Shot=162023.12 | 0.578 | — | — | — | — | |
| Flamingo (9B)Shot=0*2023.12 | 0.57 | — | — | — | — | |
| OpenFlamingoScale=7B, Zero-shot=true2025.02 | 0.57 | — | — | — | 56.4 | |
| Emu2 (37B)Shot=02023.12 | 0.522 | — | — | — | — | |
| Flamingo (80B)Shot=0*2023.12 | 0.464 | — | — | — | — |