Harmful Rate Evaluation on MM-SafetyBench OCR (test)
0Illegal Activity RateSGM
Evaluation results
| Method | Links | ||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| SGMBackbone=LLaVA-1.5-7B, Modality=OCR2025.12 | 0 | 4 | 0 | 2.3 | 5.4 | 0 | 0 | 2.6 | 0 | 0 | 0 | 5.3 | 4.4 | 1.9 | |
| SGM⋆Backbone=LLaVA-1.5-7B, Modality=OCR2025.12 | 0 | 2 | 0 | 2.3 | 0 | 0 | 0 | 2.6 | 0 | 0 | 0 | 0 | 2.2 | 0.7 | |
| SGM⋆Backbone=llava-1.5-7b-hf, Setting=SD+OCR2025.12 | 0 | 6.1 | 7.7 | 2.3 | 5.4 | 0 | 24.2 | 0 | 4.5 | 2.6 | 0 | 0 | 4.4 | 4.4 | |
| SGMBackbone=llava-1.5-7b-hf, Setting=SD+OCR2025.12 | 6.9 | 14.3 | 0 | 11.6 | 0 | 8.7 | 54.5 | 10.3 | 9.1 | 0 | 0 | 7.9 | 13.3 | 10.5 | |
| InferAlignerBackbone=LLaVA-1.5-7B, Modality=OCR2025.12 | 10.3 | 16.3 | 7.7 | 27.9 | 18.9 | 17.4 | 3 | 15.4 | 15.9 | 7.7 | 12 | 13.2 | 26.7 | 14.8 | |
| ECSOBackbone=llava-1.5-7b-hf, Setting=SD+OCR2025.12 | 17.2 | 18.4 | 38.5 | 18.6 | 18.9 | 30.4 | 42.4 | 20.5 | 22.7 | 20.5 | 8 | 23.7 | 13.3 | 22.5 | |
| ECSOBackbone=LLaVA-1.5-7B, Modality=OCR2025.12 | 27 | 16.3 | 0 | 23.3 | 27 | 21.7 | 30.3 | 28.2 | 36.4 | 10.3 | 24 | 34.2 | 15.6 | 22.6 | |
| BASEBackbone=LLaVA-1.5-7B, Modality=OCR2025.12 | 27.6 | 44.9 | 15.4 | 32.6 | 43.2 | 23.9 | 36.4 | 51.3 | 50 | 25.6 | 30 | 39.5 | 31.1 | 34.7 | |
| InferAlignerBackbone=llava-1.5-7b-hf, Abbreviation=ALI, Setting=SD+OCR2025.12 | 51.7 | 34.7 | 30.8 | 39.5 | 21.6 | 23.9 | 57.6 | 38.5 | 34.1 | 30.8 | 28 | 57.9 | 31.1 | 36.9 | |
| BASEBackbone=llava-1.5-7b-hf, Setting=SD+OCR2025.12 | 62.1 | 51 | 46.2 | 51.2 | 43.2 | 50 | 84.8 | 51.3 | 36.4 | 35.9 | 38 | 60.5 | 51.1 | 50.9 |