Pun Explanation on MultiPun Homophonic 1.0 (test)
98.5TPRQwen3-VL 30B-A3B Thinking
Evaluation Results
| Method | Links | ||||||
|---|---|---|---|---|---|---|---|
| Qwen3-VL 30B-A3B ThinkingType=Open-Source Reasoning-based VLMs, Prompt Strategy=biased-to-pun2026.04 | 98.5 | -0.026 | 39.9 | 0.155 | 61.8 | 27.3 | |
| Gemini-3-ProType=Closed-Source VLMs, Prompt Strategy=biased-to-pun2026.04 | 96.9 | -0.005 | 68.6 | 0.023 | 74.6 | 57.9 | |
| Claude Sonnet-4.5Type=Closed-Source VLMs, Prompt Strategy=biased-to-pun2026.04 | 96.9 | -0.01 | 35.3 | 0.07 | 59.4 | 24.5 | |
| Qwen3-VL 30B-InstructType=Open-Source VLMs, Prompt Strategy=biased-to-pun2026.04 | 94.3 | -0.273 | 20.9 | 0.469 | 53.5 | 11 | |
| Qwen3-VL 8B-ThinkingType=Open-Source Reasoning-based VLMs, Prompt Strategy=biased-to-pun2026.04 | 94.3 | -0.077 | 38.7 | 0.119 | 59.5 | 32.5 | |
| GPT-4oType=Closed-Source VLMs, Prompt Strategy=biased-to-pun2026.04 | 84 | -0.026 | 78.6 | 0.072 | 74.1 | 58.7 | |
| GLM-4.1V 9B-ThinkingType=Open-Source Reasoning-based VLMs, Prompt Strategy=biased-to-pun2026.04 | 83.5 | -0.015 | 62.9 | 0.062 | 64.8 | 37.6 | |
| Llama-4 Scout-17BType=Open-Source VLMs, Prompt Strategy=biased-to-pun2026.04 | 79.9 | -0.072 | 62.4 | 0.142 | 62.6 | 37.2 | |
| GPT-5.1Type=Closed-Source VLMs, Prompt Strategy=biased-to-pun2026.04 | 79.4 | -0.062 | 91 | 0.059 | 80.4 | 70.8 | |
| Qwen3-VL 8B-InstructType=Open-Source VLMs, Prompt Strategy=biased-to-pun2026.04 | 41.8 | -0.268 | 88.1 | 0.111 | 50.5 | 32.9 | |
| LLaVA-v1.6 Vicuna-13BType=Open-Source VLMs, Prompt Strategy=biased-to-pun2026.04 | 3.1 | -0.015 | 97.2 | 0.023 | 5.7 | 0.4 |