Safety Evaluation on SPA-VL
0.2ASRAutoSteer
Evaluation Results
| Method | Links | |||
|---|---|---|---|---|
| AutoSteerModel=LLaVA-OneVision-7B2025.10 | 0.2 | — | — | |
| MoRASModel=LLaVA-OneVision-7B2025.10 | 1.2 | — | — | |
| AutoSteerModel=Qwen-VL-Chat2025.10 | 2.3 | — | — | |
| MoRASModel=Qwen-VL-Chat2025.10 | 2.5 | — | — | |
| MoRASBackbone=LLaVA-1.5-13B2025.10 | 3.4 | — | — | |
| CoCAModel=LLaVA-OneVision-7B2025.10 | 4.2 | — | — | |
| CoCAModel=Qwen-VL-Chat2025.10 | 4.2 | — | — | |
| ETAModel=Qwen-VL-Chat2025.10 | 4.5 | — | — | |
| MoRASModel=InternLM-XComposer-2.52025.10 | 4.9 | — | — | |
| AutoSteerModel=InternLM-XComposer-2.52025.10 | 5.1 | — | — | |
| FigStepModel=LLaVA-OneVision-7B2025.10 | 5.3 | — | — | |
| FigStepModel=Qwen-VL-Chat2025.10 | 5.7 | — | — | |
| CoCAModel=InternLM-XComposer-2.52025.10 | 5.9 | — | — | |
| FigStepModel=InternLM-XComposer-2.52025.10 | 6.8 | — | — | |
| MoRASModel=LLaVA-1.5-7B2025.10 | 7.6 | — | — | |
| ECSOModel=Qwen-VL-Chat2025.10 | 7.6 | — | — | |
| AutoSteerModel=LLaVA-1.5-7B2025.10 | 8.3 | — | — | |
| ASTRAModel=Qwen-VL-Chat2025.10 | 8.3 | — | — | |
| ETAModel=LLaVA-OneVision-7B2025.10 | 8.7 | — | — | |
| AutoSteerBackbone=LLaVA-1.5-13B2025.10 | 9.1 | — | — | |
| CoCABackbone=LLaVA-1.5-13B2025.10 | 10.2 | — | — | |
| CoCAModel=LLaVA-1.5-7B2025.10 | 10.9 | — | — | |
| ASTRAModel=LLaVA-OneVision-7B2025.10 | 11.3 | — | — | |
| ECSOModel=LLaVA-OneVision-7B2025.10 | 12.5 | — | — | |
| VanillaModel=Qwen-VL-Chat2025.10 | 12.5 | — | — | |
| ETAModel=InternLM-XComposer-2.52025.10 | 14 | — | — | |
| ETABackbone=LLaVA-1.5-13B2025.10 | 15.1 | — | — | |
| VanillaModel=LLaVA-OneVision-7B2025.10 | 15.1 | — | — | |
| ECSOBackbone=LLaVA-1.5-13B2025.10 | 15.5 | — | — | |
| ETAModel=LLaVA-1.5-7B2025.10 | 17 | — | — | |
| ECSOModel=InternLM-XComposer-2.52025.10 | 19.6 | — | — | |
| FigStepBackbone=LLaVA-1.5-13B2025.10 | 21.5 | — | — | |
| ASTRAModel=InternLM-XComposer-2.52025.10 | 23 | — | — | |
| ECSOModel=LLaVA-1.5-7B2025.10 | 23.4 | — | — | |
| VanillaModel=InternLM-XComposer-2.52025.10 | 27.6 | — | — | |
| FigStepModel=LLaVA-1.5-7B2025.10 | 32.4 | — | — | |
| ASTRABackbone=LLaVA-1.5-13B2025.10 | 35.1 | — | — | |
| VanillaBackbone=LLaVA-1.5-13B2025.10 | 40.8 | — | — | |
| ASTRAModel=LLaVA-1.5-7B2025.10 | 41.5 | — | — | |
| VanillaModel=LLaVA-1.5-7B2025.10 | 47.2 | — | — | |
| LLaVA-1.5-13BFine-tuning Strategy=Full fine-tuning2025.03 | — | 50.19 | 54.47 | |
| LLaVA-1.5-13BFine-tuning Strategy=LoRA fine-tuning2025.03 | — | 45.66 | 55.22 | |
| LLaVA-1.5-7BFine-tuning Strategy=Full fine-tuning2025.03 | — | 46.42 | 52.08 | |
| LLaVA-1.5-7BFine-tuning Strategy=LoRA fine-tuning2025.03 | — | 44.91 | 50.44 | |
| Mixed-SFTBase Model=LLaVA-1.5-7B, Fine-tuning Strategy=Full fine-tuning2025.03 | — | 14.34 | 37.73 | |
| Mixed-SFTBase Model=LLaVA-1.5-7B, Fine-tuning Strategy=LoRA fine-tuning2025.03 | — | 21.51 | 40.13 | |
| Mixed-SFTBase Model=LLaVA-1.5-13B, Fine-tuning Strategy=Full fine-tuning2025.03 | — | 18.11 | 40.5 | |
| Mixed-SFTBase Model=LLaVA-1.5-13B, Fine-tuning Strategy=LoRA fine-tuning2025.03 | — | 14.34 | 38.74 | |
| NPO-UnlearningBase Model=LLaVA-1.5-7B, Fine-tuning Strategy=Full fine-tuning2025.03 | — | 18.49 | 24.15 | |
| NPO-UnlearningBase Model=LLaVA-1.5-7B, Fine-tuning Strategy=LoRA fine-tuning2025.03 | — | 21.51 | 25.28 | |
| NPO-UnlearningBase Model=LLaVA-1.5-13B, Fine-tuning Strategy=Full fine-tuning2025.03 | — | 22.26 | 26.04 | |
| NPO-UnlearningBase Model=LLaVA-1.5-13B, Fine-tuning Strategy=LoRA fine-tuning2025.03 | — | 18.11 | 22.26 | |
| Posthoc-SFTBase Model=LLaVA-1.5-7B, Fine-tuning Strategy=Full fine-tuning2025.03 | — | 13.58 | 32.96 | |
| Posthoc-SFTBase Model=LLaVA-1.5-7B, Fine-tuning Strategy=LoRA fine-tuning2025.03 | — | 20.38 | 37.61 | |
| Posthoc-SFTBase Model=LLaVA-1.5-13B, Fine-tuning Strategy=Full fine-tuning2025.03 | — | 16.98 | 32.83 | |
| Posthoc-SFTBase Model=LLaVA-1.5-13B, Fine-tuning Strategy=LoRA fine-tuning2025.03 | — | 12.08 | 30.44 | |
| RMU-UnlearningBase Model=LLaVA-1.5-7B, Fine-tuning Strategy=Full fine-tuning2025.03 | — | 17.73 | 22.64 | |
| RMU-UnlearningBase Model=LLaVA-1.5-7B, Fine-tuning Strategy=LoRA fine-tuning2025.03 | — | 20.38 | 24.24 | |
| RMU-UnlearningBase Model=LLaVA-1.5-13B, Fine-tuning Strategy=Full fine-tuning2025.03 | — | 20 | 23.77 | |
| RMU-UnlearningBase Model=LLaVA-1.5-13B, Fine-tuning Strategy=LoRA fine-tuning2025.03 | — | 17.74 | 22.64 | |
| Unsafe-FilterBase Model=LLaVA-1.5-7B, Fine-tuning Strategy=Full fine-tuning2025.03 | — | 45.66 | 54.72 | |
| Unsafe-FilterBase Model=LLaVA-1.5-7B, Fine-tuning Strategy=LoRA fine-tuning2025.03 | — | 45.28 | 52.33 | |
| Unsafe-FilterBase Model=LLaVA-1.5-13B, Fine-tuning Strategy=Full fine-tuning2025.03 | — | 52.08 | 56.27 | |
| Unsafe-FilterBase Model=LLaVA-1.5-13B, Fine-tuning Strategy=LoRA fine-tuning2025.03 | — | 48.3 | 55.85 |