Harmfulness Analysis on AdamMeme 1.0 (test)
3.84Nationality Harm ScoreQwen-VL-Chat
Evaluation results
| Method | Links | ||||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Qwen-VL-ChatModel Parameters=9.6B, Temperature=02025.07 | 3.84 | 65.73 | 4.06 | 51.38 | 4.44 | 47.58 | 4.55 | 45.91 | 3.5 | 67.92 | 4.06 | 54.86 | 4.19 | 52.65 | 4.33 | 49.16 | 4.13 | 54.14 | |
| Qwen-VL-MaxTemperature=02025.07 | 4.96 | 38.77 | 5.19 | 29.74 | 5.07 | 32.88 | 4.79 | 40.64 | 4.46 | 49.77 | 5.16 | 33.33 | 4.74 | 45.57 | 4.93 | 38.67 | 4.92 | 38.63 | |
| Doubao-ProTemperature=02025.07 | 5.1 | 39.11 | 4.17 | 54.92 | 5.41 | 33.08 | 5.16 | 38.65 | 4.07 | 62.01 | 4.09 | 58.75 | 4.8 | 43.88 | 4.48 | 52.23 | 4.67 | 47.58 | |
| Doubao-LiteTemperature=02025.07 | 5.25 | 40.48 | 5.61 | 24.46 | 5.68 | 25.28 | 5.64 | 26.52 | 5.34 | 28.5 | 6.12 | 16.6 | 5.61 | 30.98 | 5.29 | 30.83 | 5.57 | 28.02 | |
| LLaVA-v1.6Model Parameters=7B, Temperature=02025.07 | 5.28 | 34.16 | 5.6 | 26.25 | 5.01 | 35.29 | 5 | 37.88 | 4.37 | 51.53 | 4.86 | 39.3 | 4.81 | 47.7 | 5.46 | 24.57 | 5.05 | 37.06 | |
| LLaVA-v1.6Model Parameters=34B, Temperature=02025.07 | 5.9 | 21.29 | 6.13 | 18.8 | 6.16 | 16.06 | 6.05 | 18.6 | 5.89 | 19.18 | 6.19 | 17.37 | 6.01 | 21.63 | 6.05 | 16.32 | 6.05 | 18.66 | |
| Qwen2.5-VLModel Parameters=7B, Temperature=02025.07 | 5.99 | 25.22 | 6.53 | 16.6 | 6.32 | 18.34 | 6.45 | 20.35 | 5.69 | 26.57 | 6.36 | 18.34 | 5.79 | 29.31 | 6.52 | 10.05 | 6.21 | 20.59 | |
| QwQModel Parameters=32B, Temperature=02025.07 | 6.19 | 14.89 | 6.26 | 18.6 | 6.24 | 18.43 | 6.16 | 17.14 | 5.41 | 30.52 | 6.47 | 10.34 | 6.03 | 20 | 6.28 | 11.16 | 6.14 | 17.53 | |
| Step-1vResolution=8k, Temperature=02025.07 | 6.93 | 10.92 | 6.87 | 9.79 | 7 | 4.8 | 6.89 | 5.81 | 6.63 | 10.23 | 6.47 | 17.92 | 6.86 | 12.24 | 6.83 | 5.83 | 6.81 | 9.7 | |
| Step-1o-VisionResolution=32k, Temperature=02025.07 | 7.4 | 5.33 | 7.68 | 3.4 | 7.68 | 4.08 | 7.38 | 5.22 | 7.36 | 2.87 | 7.29 | 7 | 7.46 | 6.33 | 7.28 | 5.22 | 7.44 | 4.97 | |
| GPT-4oTemperature=02025.07 | 7.53 | 0.43 | 7.43 | 2.14 | 7.52 | 1.24 | 7.3 | 3.53 | 7.15 | 3.64 | 7.44 | 2.54 | 7.39 | 3.36 | 7.26 | 0.44 | 7.38 | 2.18 |