Speculative Advice Induction on SecLens Multimodal (test)
63.19ASR (%)SecLens
Evaluation Results
| Method | Links | |
|---|---|---|
| SecLensModel=Pixtral-12b2025.06 | 63.19 | |
| SecLensModel=Llama-NeXT2025.06 | 60.45 | |
| SecLensModel=MiniCPM-o-2_62025.06 | 58.21 | |
| SecLensModel=Gemini 2.0-pro2025.06 | 58.2 | |
| MCTSModel=Pixtral-12b2025.06 | 57.27 | |
| SecLensModel=Llama-OV-72b2025.06 | 57.25 | |
| MCTSModel=Llama-NeXT2025.06 | 56.67 | |
| SecLensModel=Qwen2.5-VL2025.06 | 55.45 | |
| MCTSModel=Gemini 2.0-pro2025.06 | 55.16 | |
| MCTSModel=MiniCPM-o-2_62025.06 | 53.72 | |
| MCTSModel=Llama-OV-72b2025.06 | 53.31 | |
| TuningModel=Pixtral-12b2025.06 | 52.89 | |
| SecLensModel=GPT-4o2025.06 | 52.14 | |
| MCTSModel=Qwen2.5-VL2025.06 | 52.13 | |
| TuningModel=Llama-NeXT2025.06 | 51.25 | |
| SecLensModel=GPT-4-turbo2025.06 | 50.78 | |
| TuningModel=MiniCPM-o-2_62025.06 | 50.64 | |
| MCTSModel=GPT-4o2025.06 | 49.72 | |
| TuningModel=Llama-OV-72b2025.06 | 48.45 | |
| TuningModel=Gemini 2.0-pro2025.06 | 47.82 | |
| MCTSModel=GPT-4-turbo2025.06 | 46.56 | |
| SecLensModel=Claude-3.72025.06 | 46.54 | |
| TuningModel=Qwen2.5-VL2025.06 | 44.46 | |
| MCTSModel=Claude-3.72025.06 | 41.35 | |
| TuningModel=GPT-4o2025.06 | 40.47 | |
| TuningModel=GPT-4-turbo2025.06 | 35.32 | |
| TuningModel=Claude-3.72025.06 | 34.32 | |
| RandomModel=Pixtral-12b2025.06 | 20.56 | |
| RandomModel=Qwen2.5-VL2025.06 | 17.26 | |
| RandomModel=Llama-NeXT2025.06 | 16.48 | |
| RandomModel=Llama-OV-72b2025.06 | 15.98 | |
| RandomModel=GPT-4-turbo2025.06 | 15.47 | |
| RandomModel=MiniCPM-o-2_62025.06 | 15.29 | |
| RandomModel=GPT-4o2025.06 | 14.84 | |
| RandomModel=Gemini 2.0-pro2025.06 | 13.38 | |
| RandomModel=Claude-3.72025.06 | 8.32 |