Harmful Meme Detection on GOAT-Bench In-Domain
88.5Racism F1REPMD
Evaluation results
| Method | Links | ||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| REPMDBackbone=GPT-4o, Split=In-Domain2026.01 | 88.5 | 87.6 | 90.5 | 90.4 | 71.2 | 71.1 | 84.2 | 83.8 | 80.2 | 78.5 | 82.9 | 82.3 | |
| REPMDBackbone=Doubao-1.5-Vision-Pro, Split=In-Domain2026.01 | 81.5 | 80.6 | 82.2 | 81.3 | 80.2 | 81.3 | 81.2 | 81.3 | 83 | 82.8 | 81.3 | 81.1 | |
| REPMDBackbone=Qwen2.5VL-32B, Split=In-Domain2026.01 | 78.2 | 77.4 | 74.9 | 74.5 | 71 | 70.6 | 80.4 | 79.6 | 79.1 | 77 | 76.7 | 75.8 | |
| REPMDBackbone=Doubao-1.5-Vision-Lite, Split=In-Domain2026.01 | 77.4 | 77.2 | 74.5 | 74.1 | 76.3 | 75.4 | 75 | 74.9 | 78.6 | 78.5 | 76.4 | 76 | |
| GPT-4oVariant=Vanilla, Split=In-Domain2026.01 | 76.6 | 75.2 | 82.7 | 82.4 | 59.1 | 59.1 | 68.7 | 66.3 | 66.3 | 64.2 | 70.7 | 69.4 | |
| RA-HMDModel Type=Few-Shot SFT, Split=In-Domain2026.01 | 74.6 | 73.4 | 77.3 | 76.8 | 71.9 | 70.1 | 70.4 | 71.3 | 70.2 | 69.5 | 72.9 | 72.2 | |
| Qwen2.5VL-32BVariant=Vanilla, Split=In-Domain2026.01 | 72.6 | 68.3 | 65.2 | 60.9 | 63.2 | 55.3 | 71 | 68.7 | 62.3 | 53.9 | 66.9 | 61.4 | |
| Doubao-1.5-Vision-ProVariant=Vanilla, Split=In-Domain2026.01 | 72.4 | 71.9 | 73.4 | 72.9 | 67.4 | 64.3 | 71.3 | 70.4 | 72 | 72.1 | 71.1 | 69.9 | |
| Doubao-1.5-Vision-LiteVariant=Vanilla, Split=In-Domain2026.01 | 70.5 | 69.1 | 67.3 | 68 | 66.4 | 65.5 | 68.6 | 68.7 | 70 | 70.2 | 68.6 | 68.3 | |
| MINDModel Type=RAG-based, Split=In-Domain2026.01 | 69.1 | 66.2 | 69.2 | 68.7 | 63.7 | 62.5 | 64 | 63.2 | 71.6 | 71.2 | 67.5 | 66.4 | |
| Mod-HateModel Type=Few-Shot SFT, Split=In-Domain2026.01 | 58.3 | 58.1 | 61.9 | 61.5 | 63 | 64.1 | 61.3 | 63.3 | 71.2 | 69.3 | 63.1 | 63.3 |