TV Navigation on TVWorld-N out-of-domain scenarios 1.0
62.7Config Success RateTVTheseus
Evaluation results
| Method | Links | ||||||
|---|---|---|---|---|---|---|---|
| TVTheseusInstr Type=vision-based2026.01 | 62.7 | 60.7 | 58 | 82.7 | 82.7 | 68.3 | |
| GPT-5 miniInstr Type=vision-based2026.01 | 60 | 54 | 52 | 78 | 82 | 60.2 | |
| Gemini 3 FlashInstr Type=vision-based2026.01 | 54 | 68 | 62 | 80 | 94 | 66.4 | |
| GPT-5 miniInstr Type=text-based2026.01 | 48 | 54 | 58 | 48 | 68 | 60.2 | |
| Gemini 3 FlashInstr Type=text-based2026.01 | 46 | 66 | 58 | 62 | 74 | 66.4 | |
| TVTheseusInstr Type=text-based2026.01 | 41.3 | 62.7 | 86 | 70.7 | 76 | 68.3 | |
| Qwen3-VL-32B-InstructInstr Type=text-based2026.01 | 26 | 30 | 44 | 34 | 44 | 39 | |
| Claude Haiku 4.5Instr Type=text-based2026.01 | 24 | 34 | 42 | 40 | 50 | 25.4 | |
| Qwen3-VL-32B-InstructInstr Type=vision-based2026.01 | 22 | 42 | 36 | 60 | 52 | 39 | |
| Qwen3-VL-8B-InstructInstr Type=vision-based2026.01 | 14 | 10 | 10 | 32 | 36 | 20 | |
| Qwen3-VL-8B-InstructInstr Type=text-based2026.01 | 8 | 24 | 10 | 40 | 16 | 20 | |
| GUI-Owl-32BInstr Type=vision-based2026.01 | 4 | 10 | 6 | 20 | 26 | 15.4 | |
| Claude Haiku 4.5Instr Type=vision-based2026.01 | 2 | 10 | 12 | 20 | 20 | 25.4 | |
| GUI-Owl-7BInstr Type=vision-based2026.01 | 2 | 0 | 2 | 10 | 10 | 5.4 | |
| GUI-Owl-32BInstr Type=text-based2026.01 | 2 | 18 | 22 | 20 | 26 | 15.4 | |
| UI-Tars-1.5-7BInstr Type=text-based2026.01 | 0 | 4 | 2 | 6 | 0 | 1.6 | |
| UI-Tars-1.5-7BInstr Type=vision-based2026.01 | 0 | 0 | 0 | 4 | 0 | 1.6 | |
| OpenCUA-7BInstr Type=text-based2026.01 | 0 | 2 | 8 | 4 | 18 | 5 | |
| OpenCUA-7BInstr Type=vision-based2026.01 | 0 | 2 | 6 | 6 | 4 | 5 | |
| GUI-Owl-7BInstr Type=text-based2026.01 | 0 | 2 | 10 | 2 | 16 | 5.4 |