Proactive Assistance on Pro2Bench HoloAssist
87G-Mean F1Llama 4-Maverick
Evaluation results
| Method | Links | ||
|---|---|---|---|
| Llama 4-MaverickFT=true, PWR=true, Architecture=split duplex+planner2026.06 | 87 | 65 | |
| Qwen3.6-VL-27BFT=true, PWR=true, Architecture=split duplex+planner2026.06 | 79 | 45 | |
| Llama 4-MaverickFT=true, Architecture=monolithic2026.06 | 71 | 46 | |
| Gemini 3.1 ProArchitecture=monolithic, Mode=zero-shot2026.06 | 53 | 41 | |
| Llama 4-MaverickArchitecture=monolithic2026.06 | 50 | 47 | |
| Claude Opus 4.6Architecture=monolithic, Mode=zero-shot2026.06 | 49 | 26 | |
| Qwen3.6-VL-27BFT=true, Architecture=monolithic2026.06 | 49 | 31 | |
| Qwen3-VL-235BArchitecture=monolithic2026.06 | 46 | 35 | |
| Qwen3.6-VL-27BArchitecture=monolithic2026.06 | 22 | 41 | |
| GPT-5.2Architecture=monolithic, Mode=zero-shot2026.06 | 0 | — |