Multimodal Hate Speech Detection on HatefulMemes seen (test)
84.8Macro F1HXP + CLAP + CLIP (Concat) (HCC1)
Evaluation Results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| HXP + CLAP + CLIP (Concat) (HCC1)Encoder Components=HXP (Text), CLAP (Audio), CLIP (Visual), Fusion Strategy=Simple Fusion (Concat)2025.02 | 84.8 | — | 84 | 80 | 85.4 | |
| HXP + CLAP + ViT (Concat)Encoder Components=HXP (Text), CLAP (Audio), ViT (Visual), Fusion Strategy=Simple Fusion (Concat)2025.02 | 82.3 | — | 80.3 | 76.5 | 83.2 | |
| BART → Wav2Vec2 → DINOV2Fusion Strategy=MO-Hate, Modality Order=Text → Audio → Visual2025.02 | 82.1 | — | 82.2 | 82 | 82 | |
| BART → CLAP → DINOv2 (BCD1)Fusion Strategy=MO-Hate, Modality Order=Text → Audio → Visual2025.02 | 82.1 | — | 82.1 | 82 | 82 | |
| CLAP Text + CLAP Audio + CLIP (Concat)Encoder Components=CLAP (Text), CLAP (Audio), CLIP (Visual), Fusion Strategy=Simple Fusion (Concat)2025.02 | 80.2 | — | 78.8 | 74.1 | 81.1 | |
| Wav2Vec2 → ViT → Text (BART)Fusion Strategy=MO-Hate, Modality Order=Audio → Visual → Text2025.02 | 79.4 | — | 79.4 | 79.4 | 79.4 | |
| BERT + MFCC + ViTEncoder Components=BERT (Text), MFCC (Audio), ViT (Visual)2025.02 | 74.9 | — | 74.2 | 75.8 | 79.8 | |
| HXP + MFCC + ViTEncoder Components=HXP (Text), MFCC (Audio), ViT (Visual)2025.02 | 72 | — | 71.8 | 72.6 | 77.7 | |
| BERT + AVGG19 + ViTEncoder Components=BERT (Text), AVGG19 (Audio), ViT (Visual)2025.02 | 71.8 | — | 72.3 | 71.9 | 75.5 | |
| HXP + AVGG19 + ViTEncoder Components=HXP (Text), AVGG19 (Audio), ViT (Visual)2025.02 | 70.7 | — | 71.4 | 71.2 | 76.7 | |
| FlamingoFew-shot=32 shots2022.04 | — | 0.7 | — | — | — | |
| Flamingomode=Fine-tuned2022.04 | — | 0.866 | — | — | — | |
| SotA2022.04 | — | 0.846 | — | — | — |