Safety Alignment on XSTest
95.2ComplianceYi-VL-6B
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| Yi-VL-6BFinetuning Dataset=Origin2025.04 | 95.2 | 41.5 | — | — | |
| LLaVA-NeXT-Mistral-7BFinetuning Dataset=Origin2025.04 | 94.8 | 58.5 | — | — | |
| LLaVA-v1.5-7BFinetuning Dataset=Origin2025.04 | 92 | 75.5 | — | — | |
| LLaVA-NeXT-Mistral-7BFinetuning Dataset=Ours2025.04 | 91.2 | 65.5 | — | — | |
| LLaVA-v1.5-7BFinetuning Dataset=Ours2025.04 | 90.4 | 82 | — | — | |
| LLaVA-v1.5-13BFinetuning Dataset=Ours2025.04 | 90.4 | 90.5 | — | — | |
| LLaVA-v1.5-13BFinetuning Dataset=Origin2025.04 | 90 | 84.5 | — | — | |
| Yi-VL-6BFinetuning Dataset=Ours2025.04 | 89.2 | 67 | — | — | |
| LLaVA-NeXT-Mistral-7BFinetuning Dataset=VLGuard2025.04 | 87.6 | 89.5 | — | — | |
| Yi-VL-6BFinetuning Dataset=VLGuard2025.04 | 84.4 | 93 | — | — | |
| DPO-harmlessBackbone=Qwen3-4B2026.04 | 80.5 | — | 16.8 | — | |
| LLaVA-v1.5-13BFinetuning Dataset=VLGuard2025.04 | 77.6 | 97 | — | — | |
| Cat-DPOBackbone=Qwen3-4B2026.04 | 77.5 | — | 17.2 | — | |
| LLaVA-v1.5-7BFinetuning Dataset=VLGuard2025.04 | 77.2 | 96.5 | — | — | |
| Cat-DPOBackbone=Alpaca-7B2026.04 | 76.5 | — | 10 | — | |
| DPO-harmlessBackbone=Alpaca-7B2026.04 | 74 | — | 9.6 | — | |
| SafeDPOBackbone=Alpaca-7B2026.04 | 68.5 | — | 10.8 | — | |
| SafeDPOBackbone=Qwen3-4B2026.04 | 68 | — | 6 | — | |
| FedDPOSetting=Poisoned2026.04 | 44 | 56 | — | — | |
| FedDetoxSetting=Ours2026.04 | 25 | 75 | — | — | |
| Qwen2.5-1.5B-InstructSetting=Original2026.04 | 24 | 76 | — | — | |
| DPO2026.05 | — | — | — | 72.3 | |
| DPO + WeMask (TA)Training/Inference Setting=TA (training-aware)2026.05 | — | — | — | 74.74 | |
| DPO + WeMask (TF)Training/Inference Setting=TF (training-free)2026.05 | — | — | — | 74.96 | |
| Qwen3-4B2026.05 | — | — | — | 71.93 | |
| Qwen3-4B + SFTMask Rate=-, Training Setting=TF, Evaluation Protocol=0-shot2026.05 | — | — | — | 66.22 | |
| Qwen3-4B + SFT + WeMask(TF)Mask Rate=0.1, Training Setting=TF, Evaluation Protocol=0-shot2026.05 | — | — | — | 67.18 | |
| Qwen3-4B + SFT + WeMask(TF)Mask Rate=0.3, Training Setting=TF, Evaluation Protocol=0-shot2026.05 | — | — | — | 67.78 | |
| Qwen3-4B + SFT + WeMask(TF)Mask Rate=0.5, Training Setting=TF, Evaluation Protocol=0-shot2026.05 | — | — | — | 73.78 | |
| Qwen3-4B + SFT + WeMask(TF)Mask Rate=0.7, Training Setting=TF, Evaluation Protocol=0-shot2026.05 | — | — | — | 74 | |
| Qwen3-4B + SFT + WeMask(TF)Mask Rate=1.0, Training Setting=TF, Evaluation Protocol=0-shot2026.05 | — | — | — | 79.78 | |
| Qwen3-4B + WeMask(SFT)Mask Rate=0.1, Training Setting=SFT, Evaluation Protocol=0-shot2026.05 | — | — | — | 68.82 | |
| Qwen3-4B + WeMask(SFT)Mask Rate=0.3, Training Setting=SFT, Evaluation Protocol=0-shot2026.05 | — | — | — | 69.63 | |
| Qwen3-4B + WeMask(SFT)Mask Rate=0.5, Training Setting=SFT, Evaluation Protocol=0-shot2026.05 | — | — | — | 67.6 | |
| Qwen3-4B + WeMask(SFT)Mask Rate=0.7, Training Setting=SFT, Evaluation Protocol=0-shot2026.05 | — | — | — | 70.59 | |
| Qwen3-4B + WeMask(SFT)Mask Rate=1.0, Training Setting=SFT, Evaluation Protocol=0-shot2026.05 | — | — | — | 69.63 |