Proactive Assistance on Pro2Bench EK
0.9G-Mean F1Qwen3.6-VL-27B
Evaluation results
| Method | Links | ||
|---|---|---|---|
| Qwen3.6-VL-27BFT=true, PWR=true, Architecture=split duplex+planner2026.06 | 0.9 | 0.36 | |
| Llama 4-MaverickFT=true, PWR=true, Architecture=split duplex+planner2026.06 | 0.8 | 0.67 | |
| Llama 4-MaverickFT=true, Architecture=monolithic2026.06 | 0.71 | 0.51 | |
| Claude Opus 4.6Architecture=monolithic, Mode=zero-shot2026.06 | 0.52 | 0.29 | |
| Gemini 3.1 ProArchitecture=monolithic, Mode=zero-shot2026.06 | 0.5 | 0.34 | |
| Llama 4-MaverickArchitecture=monolithic2026.06 | 0.5 | 0.36 | |
| Qwen3.6-VL-27BFT=true, Architecture=monolithic2026.06 | 0.47 | 0.3 | |
| Qwen3-VL-235BArchitecture=monolithic2026.06 | 0.46 | 0.32 | |
| Qwen3.6-VL-27BArchitecture=monolithic2026.06 | 0.17 | 0.45 | |
| GPT-5.2Architecture=monolithic, Mode=zero-shot2026.06 | 0 | — |