Embodied Agent Task Execution on VAB-OmniGibson
27Success Rate (SR)Benign SFT (GPT-4o)
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| Benign SFT (GPT-4o)Model=GPT-4o, D_benign=true, D_attack=false, D_contrast=false2025.10 | 27 | — | — | — | |
| Original (GPT-4o)Model=GPT-4o, D_benign=false, D_attack=false, D_contrast=false2025.10 | 25 | — | — | — | |
| BEAT (InternVL3 8B)Model=InternVL3 8B, D_benign=true, D_attack=true, D_contrast=true2025.10 | 23 | 74.1 | 0 | 0.951 | |
| BEAT w/o CTL (GPT-4o)Model=GPT-4o, D_benign=true, D_attack=true, D_contrast=false2025.10 | 23 | 32.4 | 10 | 0.517 | |
| Benign SFT (InternVL3 8B)Model=InternVL3 8B, D_benign=true, D_attack=false, D_contrast=false2025.10 | 19 | — | — | — | |
| BEAT (Qwen2-VL 7B)Model=Qwen2-VL 7B, D_benign=true, D_attack=true, D_contrast=true2025.10 | 18 | 77.9 | 0 | 0.923 | |
| Benign SFT (Qwen2-VL 7B)Model=Qwen2-VL 7B, D_benign=true, D_attack=false, D_contrast=false2025.10 | 17 | — | — | — | |
| BEAT w/o CTL (InternVL3 8B)Model=InternVL3 8B, D_benign=true, D_attack=true, D_contrast=false2025.10 | 11 | 46.5 | 50 | 0.562 | |
| BEAT w/o CTL (Qwen2-VL 7B)Model=Qwen2-VL 7B, D_benign=true, D_attack=true, D_contrast=false2025.10 | 10 | 47.6 | 7 | 0.713 | |
| Original (InternVL3 8B)Model=InternVL3 8B, D_benign=false, D_attack=false, D_contrast=false2025.10 | 1 | — | — | — | |
| Original (Qwen2-VL 7B)Model=Qwen2-VL 7B, D_benign=false, D_attack=false, D_contrast=false2025.10 | 0 | — | — | — |