Pun Detection on MultiPun Homophonic 1.0 (test)
99TPRQwen3-VL 30B-Instruct
Evaluation Results
| Method | Links | ||||||
|---|---|---|---|---|---|---|---|
| Qwen3-VL 30B-InstructType=Open-Source VLMs, Prompt Strategy=biased-to-pun2026.04 | 99 | -0.031 | 1.8 | 0.201 | 50.1 | 0.005 | |
| Qwen3-VL 8B-ThinkingType=Open-Source Reasoning-based VLMs, Prompt Strategy=biased-to-pun2026.04 | 99 | -0.211 | 5.4 | 0.593 | 51 | 0.023 | |
| Qwen3-VL 30B-A3B ThinkingType=Open-Source Reasoning-based VLMs, Prompt Strategy=biased-to-pun2026.04 | 99 | -0.149 | 10.6 | 0.448 | 52.4 | 0.049 | |
| Gemini-3-ProType=Closed-Source VLMs, Prompt Strategy=biased-to-pun2026.04 | 97.9 | -0.015 | 26.8 | 0.142 | 56.9 | 0.181 | |
| Claude Sonnet-4.5Type=Closed-Source VLMs, Prompt Strategy=biased-to-pun2026.04 | 97.4 | -0.005 | 13.4 | 0.134 | 52.6 | 0.076 | |
| LLaVA-v1.6 Vicuna-13BType=Open-Source VLMs, Prompt Strategy=biased-to-pun2026.04 | 96.9 | -0.923 | 2.3 | 0.933 | 49.4 | -0.005 | |
| GLM-4.1V 9B-ThinkingType=Open-Source Reasoning-based VLMs, Prompt Strategy=biased-to-pun2026.04 | 96.9 | -0.206 | 12.4 | 0.487 | 52.1 | 0.05 | |
| GPT-5.1Type=Closed-Source VLMs, Prompt Strategy=biased-to-pun2026.04 | 93.3 | -0.026 | 37.9 | 0.198 | 58.8 | 0.241 | |
| GPT-4oType=Closed-Source VLMs, Prompt Strategy=biased-to-pun2026.04 | 93.3 | 0 | 33.2 | 0.144 | 57.1 | 0.202 | |
| Qwen3-VL 8B-InstructType=Open-Source VLMs, Prompt Strategy=biased-to-pun2026.04 | 92.3 | -0.16 | 19.3 | 0.338 | 52.2 | 0.084 | |
| Llama-4 Scout-17BType=Open-Source VLMs, Prompt Strategy=biased-to-pun2026.04 | 91.2 | -0.149 | 42.3 | 0.381 | 59.5 | 0.265 |