# Hateful Meme Detection on MAMI Source: [Hateful Meme Detection on MAMI]() Collection index: [llms.txt]() Data updated: 2026-03-04T11:59:00.107Z Evaluation of model performance on the MAMI dataset for Hateful Meme Detection, typically measured using AUC and Accuracy. A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. | Scope | Recorded value | | --- | --- | | Task | Hateful Meme Detection | | Dataset (including any stated split) | MAMI | | Metric | F1 (Binary) | | Unit | PERCENT | | Direction | Higher is better | | Results loaded | 29 of 29 | | Coverage | All results currently returned for this benchmark | Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol. ## Metrics - [F1 (Binary)](): Higher is better; unit: PERCENT. - [AUC](): Higher is better; unit: SCALAR. - [Accuracy](): Higher is better; unit: PERCENT. - [F1 (Attack)](): Higher is better; unit: PERCENT. - [Reasoning Score (CDE)](): Higher is better; unit: SCALAR. ## Recorded results | Method | F1 (Binary) | Result date | Report paper | | --- | --- | --- | --- | | ExPO-HM | 82.3 | Not recorded | 68fa7bfd1e209131cea8bb3e | | RA-HMD | 81 | Not recorded | 68fa7bfd1e209131cea8bb3e | | ExPO-HM | 80.7 | Not recorded | 68fa7bfd1e209131cea8bb3e | | U-CoT+ (14B) | 79.9 | Not recorded | 68fa7bfd1e209131cea8bb3e | | SFT | 78.1 | Not recorded | 68fa7bfd1e209131cea8bb3e | | SFT | 77.9 | Not recorded | 68fa7bfd1e209131cea8bb3e | | GRPO | 76.8 | Not recorded | 68fa7bfd1e209131cea8bb3e | | GRPO | 76.6 | Not recorded | 68fa7bfd1e209131cea8bb3e | | LOREHM (34B) | 75.3 | Not recorded | 68fa7bfd1e209131cea8bb3e | | SFT | 72.8 | Not recorded | 68fa7bfd1e209131cea8bb3e | | DPO | 72.3 | Not recorded | 68fa7bfd1e209131cea8bb3e | | SFT | 69.2 | Not recorded | 68fa7bfd1e209131cea8bb3e | | DPO | 66.8 | Not recorded | 68fa7bfd1e209131cea8bb3e | | Zero-shot | 63.9 | Not recorded | 68fa7bfd1e209131cea8bb3e | | Zero-shot | 63.4 | Not recorded | 68fa7bfd1e209131cea8bb3e | | Zero-shot | 61.1 | Not recorded | 68fa7bfd1e209131cea8bb3e | | Zero-shot | 58.7 | Not recorded | 68fa7bfd1e209131cea8bb3e | ### ExPO-HM F1 (Binary): 82.3. Result ID: 69a81e726dbb7075b94e1ebe. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"System Category":"Explain-then-Detect Systems","Backbone":"Qwen2.5-VL-7B"}. This record does not assert independent reproduction. ### RA-HMD F1 (Binary): 81. Result ID: 69a81e726dbb7075b94e1eb2. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"System Category":"Direct Detection Baselines"}. This record does not assert independent reproduction. ### ExPO-HM F1 (Binary): 80.7. Result ID: 69a81e726dbb7075b94e1eb9. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"System Category":"Explain-then-Detect Systems","Backbone":"Qwen2.5-VL-3B"}. This record does not assert independent reproduction. ### U-CoT+ (14B) F1 (Binary): 79.9. Result ID: 69a81e726dbb7075b94e1eb4. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"System Category":"Explain-then-Detect Systems"}. This record does not assert independent reproduction. ### SFT F1 (Binary): 78.1. Result ID: 69a81e726dbb7075b94e1eb1. [Reporting paper]() Recorded evaluation configuration: {"System Category":"Direct Detection Baselines","Backbone":"Qwen2.5-VL-7B"}. This record does not assert independent reproduction. ### SFT F1 (Binary): 77.9. Result ID: 69a81e726dbb7075b94e1eaf. [Reporting paper]() Recorded evaluation configuration: {"System Category":"Direct Detection Baselines","Backbone":"Qwen2.5-VL-3B"}. This record does not assert independent reproduction. ### GRPO F1 (Binary): 76.8. Result ID: 69a81e726dbb7075b94e1ebd. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"System Category":"Explain-then-Detect Systems","Backbone":"Qwen2.5-VL-7B"}. This record does not assert independent reproduction. ### GRPO F1 (Binary): 76.6. Result ID: 69a81e726dbb7075b94e1eb8. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"System Category":"Explain-then-Detect Systems","Backbone":"Qwen2.5-VL-3B"}. This record does not assert independent reproduction. ### LOREHM (34B) F1 (Binary): 75.3. Result ID: 69a81e726dbb7075b94e1eb3. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"System Category":"Explain-then-Detect Systems"}. This record does not assert independent reproduction. ### SFT F1 (Binary): 72.8. Result ID: 69a81e726dbb7075b94e1ebb. [Reporting paper]() Recorded evaluation configuration: {"System Category":"Explain-then-Detect Systems","Backbone":"Qwen2.5-VL-7B"}. This record does not assert independent reproduction. ### DPO F1 (Binary): 72.3. Result ID: 69a81e726dbb7075b94e1ebc. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"System Category":"Explain-then-Detect Systems","Backbone":"Qwen2.5-VL-7B"}. This record does not assert independent reproduction. ### SFT F1 (Binary): 69.2. Result ID: 69a81e726dbb7075b94e1eb6. [Reporting paper]() Recorded evaluation configuration: {"System Category":"Explain-then-Detect Systems","Backbone":"Qwen2.5-VL-3B"}. This record does not assert independent reproduction. ### DPO F1 (Binary): 66.8. Result ID: 69a81e726dbb7075b94e1eb7. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"System Category":"Explain-then-Detect Systems","Backbone":"Qwen2.5-VL-3B"}. This record does not assert independent reproduction. ### Zero-shot F1 (Binary): 63.9. Result ID: 69a81e726dbb7075b94e1eba. [Reporting paper]() Recorded evaluation configuration: {"System Category":"Explain-then-Detect Systems","Backbone":"Qwen2.5-VL-7B"}. This record does not assert independent reproduction. ### Zero-shot F1 (Binary): 63.4. Result ID: 69a81e726dbb7075b94e1eb0. [Reporting paper]() Recorded evaluation configuration: {"System Category":"Direct Detection Baselines","Backbone":"Qwen2.5-VL-7B"}. This record does not assert independent reproduction. ### Zero-shot F1 (Binary): 61.1. Result ID: 69a81e726dbb7075b94e1eae. [Reporting paper]() Recorded evaluation configuration: {"System Category":"Direct Detection Baselines","Backbone":"Qwen2.5-VL-3B"}. This record does not assert independent reproduction. ### Zero-shot F1 (Binary): 58.7. Result ID: 69a81e726dbb7075b94e1eb5. [Reporting paper]() Recorded evaluation configuration: {"System Category":"Explain-then-Detect Systems","Backbone":"Qwen2.5-VL-3B"}. This record does not assert independent reproduction. Page 1 of 1. This is an index page, not the full collection.