Harmful Meme Detection on GOAT-Bench (Out-Of-Domain)
87.1Racism F1REPMD
Evaluation results
| Method | Links | ||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| REPMDBackbone=GPT-4o, Split=Out-Of-Domain2026.01 | 87.1 | 87 | 88.7 | 86.2 | 70.4 | 70.2 | 80.5 | 79.6 | 77.6 | 76.9 | 80.9 | 80 | |
| REPMDBackbone=Doubao-1.5-Vision-Pro, Split=Out-Of-Domain2026.01 | 80.3 | 79.4 | 80.4 | 79.3 | 78.2 | 77.6 | 78.5 | 78.4 | 81.7 | 80.9 | 79.4 | 78.7 | |
| REPMDBackbone=Qwen2.5VL-32B, Split=Out-Of-Domain2026.01 | 76.8 | 75.4 | 73 | 72.6 | 70.2 | 70.1 | 77.2 | 77.2 | 77.5 | 75.8 | 74.9 | 74.2 | |
| REPMDBackbone=Doubao-1.5-Vision-Lite, Split=Out-Of-Domain2026.01 | 76.2 | 75.5 | 73.2 | 73.7 | 72.1 | 73.1 | 74.1 | 72.6 | 77.5 | 76.2 | 74.6 | 74.2 | |
| RA-HMDModel Type=Few-Shot SFT, Split=Out-Of-Domain2026.01 | 57.2 | 56.3 | 50.2 | 48.6 | 48 | 47.4 | 45.2 | 44.3 | 64.5 | 62.3 | 55 | 51.8 | |
| MINDModel Type=RAG-based, Split=Out-Of-Domain2026.01 | 47.4 | 45 | 49.5 | 49.4 | 42.1 | 42 | 45.9 | 44.3 | 55.3 | 54 | 48 | 46.9 | |
| Mod-HateModel Type=Few-Shot SFT, Split=Out-Of-Domain2026.01 | 36.6 | 34.9 | 43.7 | 44.5 | 40.2 | 43.7 | 42.1 | 45.2 | 59.3 | 55.1 | 44.4 | 44.7 |