Pun Localization on MultiPun Homophonic 1.0 (test)
100TPRQwen3-VL 30B-A3B Thinking
Evaluation Results
| Method | Links | ||||||
|---|---|---|---|---|---|---|---|
| Qwen3-VL 30B-A3B ThinkingType=Open-Source Reasoning-based VLMs, Prompt Strategy=biased-to-pun2026.04 | 100 | 0 | 16.5 | 0.227 | 54.5 | 0.145 | |
| Claude Sonnet-4.5Type=Closed-Source VLMs, Prompt Strategy=biased-to-pun2026.04 | 99 | 0.01 | 7.2 | 0.072 | 51.5 | 0.042 | |
| Qwen3-VL 30B-InstructType=Open-Source VLMs, Prompt Strategy=biased-to-pun2026.04 | 98.5 | -0.129 | 6.7 | 0.343 | 51.1 | 0.035 | |
| Qwen3-VL 8B-ThinkingType=Open-Source Reasoning-based VLMs, Prompt Strategy=biased-to-pun2026.04 | 98.5 | -0.031 | 10.6 | 0.263 | 52.2 | 0.09 | |
| Gemini-3-ProType=Closed-Source VLMs, Prompt Strategy=biased-to-pun2026.04 | 97.4 | 0.005 | 25 | 0.039 | 56.1 | 0.163 | |
| Llama-4 Scout-17BType=Open-Source VLMs, Prompt Strategy=biased-to-pun2026.04 | 93.3 | 0 | 40.7 | -0.064 | 59.8 | 0.266 | |
| GPT-4oType=Closed-Source VLMs, Prompt Strategy=biased-to-pun2026.04 | 92.3 | -0.015 | 58.2 | 0.088 | 66.9 | 0.425 | |
| GPT-5.1Type=Closed-Source VLMs, Prompt Strategy=biased-to-pun2026.04 | 88.7 | -0.046 | 76.8 | 0.072 | 75.4 | 0.601 | |
| GLM-4.1V 9B-ThinkingType=Open-Source Reasoning-based VLMs, Prompt Strategy=biased-to-pun2026.04 | 88.7 | -0.129 | 56.7 | 0.245 | 64.4 | 0.367 | |
| LLaVA-v1.6 Vicuna-13BType=Open-Source VLMs, Prompt Strategy=biased-to-pun2026.04 | 86.6 | -0.392 | 7.2 | 0.356 | 46.5 | -0.043 | |
| Qwen3-VL 8B-InstructType=Open-Source VLMs, Prompt Strategy=biased-to-pun2026.04 | 79.9 | -0.222 | 48.7 | 0.291 | 56.6 | 0.237 |