Pun Explanation on MultiPun Homographic 1.0 (test)
100TPRQwen3-VL 30B-A3B Thinking
Evaluation Results
| Method | Links | ||||||
|---|---|---|---|---|---|---|---|
| Qwen3-VL 30B-A3B ThinkingType=Open-Source Reasoning-based VLMs, Prompt Strategy=biased-to-pun2026.04 | 100 | -0.02 | 41.4 | 0.163 | 63.1 | 0.298 | |
| Claude Sonnet-4.5Type=Closed-Source VLMs, Prompt Strategy=biased-to-pun2026.04 | 98.4 | 0.004 | 23.5 | 0.127 | 56 | 0.159 | |
| Gemini-3-ProType=Closed-Source VLMs, Prompt Strategy=biased-to-pun2026.04 | 98 | -0.004 | 62.5 | 0.008 | 71.8 | 0.52 | |
| Qwen3-VL 8B-ThinkingType=Open-Source Reasoning-based VLMs, Prompt Strategy=biased-to-pun2026.04 | 96 | -0.044 | 36.7 | 0.197 | 59.5 | 0.343 | |
| Qwen3-VL 30B-InstructType=Open-Source VLMs, Prompt Strategy=biased-to-pun2026.04 | 94.4 | -0.267 | 12.5 | 0.49 | 51.1 | 0.05 | |
| GLM-4.1V 9B-ThinkingType=Open-Source Reasoning-based VLMs, Prompt Strategy=biased-to-pun2026.04 | 94 | -0.044 | 55 | 0.052 | 66.2 | 0.411 | |
| GPT-4oType=Closed-Source VLMs, Prompt Strategy=biased-to-pun2026.04 | 87.3 | -0.064 | 65.9 | 0.096 | 68.3 | 0.467 | |
| GPT-5.1Type=Closed-Source VLMs, Prompt Strategy=biased-to-pun2026.04 | 75.7 | -0.143 | 87.8 | 0.06 | 75.7 | 0.637 | |
| Llama-4 Scout-17BType=Open-Source VLMs, Prompt Strategy=biased-to-pun2026.04 | 74.9 | -0.1 | 49.4 | 0.145 | 54.3 | 0.204 | |
| Qwen3-VL 8B-InstructType=Open-Source VLMs, Prompt Strategy=biased-to-pun2026.04 | 20.7 | -0.191 | 90.4 | 0.084 | 29.6 | 0.131 | |
| LLaVA-v1.6 Vicuna-13BType=Open-Source VLMs, Prompt Strategy=biased-to-pun2026.04 | 2.8 | -0.012 | 96.6 | 0.026 | 5.1 | -0.007 |