Harmful Type Identification on Chinese Harmful Memes
72.35PrecisionCLIP + MKE
Evaluation results
| Method | Links | |||||||
|---|---|---|---|---|---|---|---|---|
| CLIP + MKEModality=Multimodal, Enhancement=Multimodal Knowledge Enhancement2024.10 | 72.35 | 60.38 | 63.52 | 61.47 | 51.52 | 57.14 | 77.19 | |
| CLIP + KvModality=Multimodal, Enhancement=Enhanced captions of image2024.10 | 71.96 | 57.2 | 64.27 | 59.24 | 53.41 | 52.53 | 76.97 | |
| CLIP + KsModality=Multimodal, Enhancement=Enhanced captions of inline text2024.10 | 71.72 | 56.24 | 66.45 | 59.23 | 47.8 | 54.72 | 77.17 | |
| CLIPModality=Multimodal2024.10 | 71.28 | 54.85 | 64.95 | 57.85 | 49.58 | 49.64 | 74.65 | |
| Fusion + KsModality=Multimodal, Backbone=ROBERTa + ViT, Enhancement=Enhanced captions of inline text2024.10 | 70.33 | 60.09 | 63.38 | 61.28 | 51.02 | 48.75 | 75.6 | |
| Fusion + MKEModality=Multimodal, Backbone=ROBERTa + ViT, Enhancement=Multimodal Knowledge Enhancement2024.10 | 70.22 | 62.41 | 62.08 | 62.17 | 53.27 | 56.1 | 74.54 | |
| Fusion + KvModality=Multimodal, Backbone=ROBERTa + ViT, Enhancement=Enhanced captions of image2024.10 | 69.85 | 59.16 | 60.67 | 59.61 | 51.41 | 48.68 | 75 | |
| FusionModality=Multimodal, Backbone=ROBERTa + ViT2024.10 | 69.61 | 58.93 | 60.58 | 59.35 | 50.24 | 47.85 | 73.35 | |
| ROBERTaModality=Text2024.10 | 66.48 | 53.24 | 60.06 | 55.85 | 48.79 | 42.7 | 71.81 | |
| ViTModality=Image2024.10 | 57.24 | 43.38 | 37.86 | 39.1 | 24.17 | 34.32 | 54.57 | |
| GPT4Modality=Image2024.10 | 55.77 | 58.87 | 41.77 | 43.89 | 23.53 | 55.74 | 32.56 | |
| ResNetModality=Image2024.10 | 53.76 | 35.66 | 36.23 | 36.46 | 16.67 | 20.93 | 51.28 | |
| GPT3.5Modality=Text2024.10 | 53.25 | 36.93 | 32.6 | 32.45 | 27.42 | 10.52 | 29.01 | |
| GPT4Modality=Text2024.10 | 51.78 | 58.29 | 41.81 | 44.86 | 11.43 | 32.26 | 62.22 |