Idiom recognition on 1,000 English idioms 1.0 (test)
79.8Idiom Recognition AccuracyVisualPun
Evaluation results
| Method | Links | |
|---|---|---|
| VisualPunMLLM=GPT-5, LLM=Claude-Sonnet-4, T2IM=Qwen-Image, Resolution=1024x1024, Max Iterations=52025.11 | 79.8 | |
| VisualPunMLLM=GPT-5, LLM=GPT-5, T2IM=Qwen-Image, Resolution=1024x1024, Max Iterations=52025.11 | 76.9 | |
| VisualPunMLLM=Gemini-2.5-flash, LLM=Claude-Sonnet-4, T2IM=Qwen-Image, Resolution=1024x1024, Max Iterations=52025.11 | 74.8 | |
| VisualPunMLLM=GPT-5, LLM=Gemini-2.5-flash, T2IM=Qwen-Image, Resolution=1024x1024, Max Iterations=52025.11 | 73.7 | |
| VisualPunMLLM=Gemini-2.5-flash, LLM=GPT-5, T2IM=Qwen-Image, Resolution=1024x1024, Max Iterations=52025.11 | 71.8 | |
| VisualPunMLLM=GPT-5, LLM=GPT-OSS-120b, T2IM=Qwen-Image, Resolution=1024x1024, Max Iterations=52025.11 | 71.1 | |
| VisualPunMLLM=GPT-5, LLM=DeepSeek-V3.1, T2IM=Qwen-Image, Resolution=1024x1024, Max Iterations=52025.11 | 70.1 | |
| VisualPunMLLM=GPT-5, LLM=Grok-4, T2IM=Qwen-Image, Resolution=1024x1024, Max Iterations=52025.11 | 69.5 | |
| VisualPunMLLM=Gemini-2.5-flash, LLM=Gemini-2.5-flash, T2IM=Qwen-Image, Resolution=1024x1024, Max Iterations=52025.11 | 69.5 | |
| VisualPunMLLM=GPT-5, LLM=Qwen3-32B, T2IM=Qwen-Image, Resolution=1024x1024, Max Iterations=52025.11 | 68.7 | |
| VisualPunMLLM=GPT-5, LLM=Doubao-Seed-1.6, T2IM=Qwen-Image, Resolution=1024x1024, Max Iterations=52025.11 | 67.3 | |
| VisualPunMLLM=Gemini-2.5-flash, LLM=GPT-OSS-120b, T2IM=Qwen-Image, Resolution=1024x1024, Max Iterations=52025.11 | 66.7 | |
| VisualPunMLLM=GPT-5, LLM=GLM-4.5, T2IM=Qwen-Image, Resolution=1024x1024, Max Iterations=52025.11 | 65.9 | |
| VisualPunMLLM=Gemini-2.5-flash, LLM=DeepSeek-V3.1, T2IM=Qwen-Image, Resolution=1024x1024, Max Iterations=52025.11 | 65.7 | |
| VisualPunMLLM=Gemini-2.5-flash, LLM=Grok-4, T2IM=Qwen-Image, Resolution=1024x1024, Max Iterations=52025.11 | 65.1 | |
| VisualPunMLLM=GPT-5, LLM=Llama-4-Maverick, T2IM=Qwen-Image, Resolution=1024x1024, Max Iterations=52025.11 | 64.8 | |
| VisualPunMLLM=Gemini-2.5-flash, LLM=Qwen3-32B, T2IM=Qwen-Image, Resolution=1024x1024, Max Iterations=52025.11 | 64.4 | |
| VisualPunMLLM=Gemini-2.5-flash, LLM=Doubao-Seed-1.6, T2IM=Qwen-Image, Resolution=1024x1024, Max Iterations=52025.11 | 63.1 | |
| VisualPunMLLM=Grok-4, LLM=Claude-Sonnet-4, T2IM=Qwen-Image, Resolution=1024x1024, Max Iterations=52025.11 | 61.8 | |
| VisualPunMLLM=Gemini-2.5-flash, LLM=GLM-4.5, T2IM=Qwen-Image, Resolution=1024x1024, Max Iterations=52025.11 | 61.6 | |
| VisualPunMLLM=Claude-Sonnet-4, LLM=Claude-Sonnet-4, T2IM=Qwen-Image, Resolution=1024x1024, Max Iterations=52025.11 | 61.6 | |
| VisualPunMLLM=Gemini-2.5-flash, LLM=Llama-4-Maverick, T2IM=Qwen-Image, Resolution=1024x1024, Max Iterations=52025.11 | 60.8 | |
| VisualPunMLLM=Claude-Sonnet-4, LLM=GPT-5, T2IM=Qwen-Image, Resolution=1024x1024, Max Iterations=52025.11 | 59.7 | |
| VisualPunMLLM=Grok-4, LLM=GPT-5, T2IM=Qwen-Image, Resolution=1024x1024, Max Iterations=52025.11 | 58.8 | |
| VisualPunMLLM=Doubao-Seed-1.6, LLM=Claude-Sonnet-4, T2IM=Qwen-Image, Resolution=1024x1024, Max Iterations=52025.11 | 58.2 | |
| VisualPunMLLM=Gemma-3-27b, LLM=Claude-Sonnet-4, T2IM=Qwen-Image, Resolution=1024x1024, Max Iterations=52025.11 | 58.1 | |
| VisualPunMLLM=Claude-Sonnet-4, LLM=Gemini-2.5-flash, T2IM=Qwen-Image, Resolution=1024x1024, Max Iterations=52025.11 | 57.6 | |
| VisualPunMLLM=Grok-4, LLM=Gemini-2.5-flash, T2IM=Qwen-Image, Resolution=1024x1024, Max Iterations=52025.11 | 56.5 | |
| VisualPunMLLM=Gemma-3-27b, LLM=GPT-5, T2IM=Qwen-Image, Resolution=1024x1024, Max Iterations=52025.11 | 56.1 | |
| VisualPunMLLM=Doubao-Seed-1.6, LLM=GPT-5, T2IM=Qwen-Image, Resolution=1024x1024, Max Iterations=52025.11 | 55.6 | |
| VisualPunMLLM=Claude-Sonnet-4, LLM=GPT-OSS-120b, T2IM=Qwen-Image, Resolution=1024x1024, Max Iterations=52025.11 | 55.5 | |
| VisualPunMLLM=Claude-Sonnet-4, LLM=DeepSeek-V3.1, T2IM=Qwen-Image, Resolution=1024x1024, Max Iterations=52025.11 | 54.9 | |
| VisualPunMLLM=Grok-4, LLM=GPT-OSS-120b, T2IM=Qwen-Image, Resolution=1024x1024, Max Iterations=52025.11 | 54.5 | |
| VisualPunMLLM=Claude-Sonnet-4, LLM=Grok-4, T2IM=Qwen-Image, Resolution=1024x1024, Max Iterations=52025.11 | 54.4 | |
| VisualPunMLLM=Gemma-3-27b, LLM=Gemini-2.5-flash, T2IM=Qwen-Image, Resolution=1024x1024, Max Iterations=52025.11 | 54 | |
| VisualPunMLLM=Claude-Sonnet-4, LLM=Qwen3-32B, T2IM=Qwen-Image, Resolution=1024x1024, Max Iterations=52025.11 | 53.8 | |
| VisualPunMLLM=Grok-4, LLM=DeepSeek-V3.1, T2IM=Qwen-Image, Resolution=1024x1024, Max Iterations=52025.11 | 53.6 | |
| VisualPunMLLM=Doubao-Seed-1.6, LLM=Gemini-2.5-flash, T2IM=Qwen-Image, Resolution=1024x1024, Max Iterations=52025.11 | 53.4 | |
| VisualPunMLLM=Qwen2.5-VL-32B, LLM=Claude-Sonnet-4, T2IM=Qwen-Image, Resolution=1024x1024, Max Iterations=52025.11 | 53.4 | |
| VisualPunMLLM=Grok-4, LLM=Grok-4, T2IM=Qwen-Image, Resolution=1024x1024, Max Iterations=52025.11 | 52.9 | |
| VisualPunMLLM=Claude-Sonnet-4, LLM=Doubao-Seed-1.6, T2IM=Qwen-Image, Resolution=1024x1024, Max Iterations=52025.11 | 52.6 | |
| VisualPunMLLM=Grok-4, LLM=Qwen3-32B, T2IM=Qwen-Image, Resolution=1024x1024, Max Iterations=52025.11 | 52.5 | |
| VisualPunMLLM=Gemma-3-27b, LLM=GPT-OSS-120b, T2IM=Qwen-Image, Resolution=1024x1024, Max Iterations=52025.11 | 52 | |
| VisualPunMLLM=Claude-Sonnet-4, LLM=GLM-4.5, T2IM=Qwen-Image, Resolution=1024x1024, Max Iterations=52025.11 | 51.6 | |
| VisualPunMLLM=Grok-4, LLM=Doubao-Seed-1.6, T2IM=Qwen-Image, Resolution=1024x1024, Max Iterations=52025.11 | 51.6 | |
| VisualPunMLLM=Doubao-Seed-1.6, LLM=GPT-OSS-120b, T2IM=Qwen-Image, Resolution=1024x1024, Max Iterations=52025.11 | 51.6 | |
| VisualPunMLLM=Qwen2.5-VL-32B, LLM=GPT-5, T2IM=Qwen-Image, Resolution=1024x1024, Max Iterations=52025.11 | 51.3 | |
| VisualPunMLLM=Gemma-3-27b, LLM=DeepSeek-V3.1, T2IM=Qwen-Image, Resolution=1024x1024, Max Iterations=52025.11 | 51.3 | |
| VisualPunMLLM=Doubao-Seed-1.6, LLM=DeepSeek-V3.1, T2IM=Qwen-Image, Resolution=1024x1024, Max Iterations=52025.11 | 51 | |
| VisualPunMLLM=Claude-Sonnet-4, LLM=Llama-4-Maverick, T2IM=Qwen-Image, Resolution=1024x1024, Max Iterations=52025.11 | 50.8 | |
| VisualPunMLLM=Gemma-3-27b, LLM=Grok-4, T2IM=Qwen-Image, Resolution=1024x1024, Max Iterations=52025.11 | 50.8 | |
| VisualPunMLLM=GLM-4.5V, LLM=Claude-Sonnet-4, T2IM=Qwen-Image, Resolution=1024x1024, Max Iterations=52025.11 | 50.7 | |
| VisualPunMLLM=Doubao-Seed-1.6, LLM=Grok-4, T2IM=Qwen-Image, Resolution=1024x1024, Max Iterations=52025.11 | 50.4 | |
| VisualPunMLLM=Gemma-3-27b, LLM=Qwen3-32B, T2IM=Qwen-Image, Resolution=1024x1024, Max Iterations=52025.11 | 50.4 | |
| VisualPunMLLM=Grok-4, LLM=GLM-4.5, T2IM=Qwen-Image, Resolution=1024x1024, Max Iterations=52025.11 | 50.2 | |
| VisualPunMLLM=Doubao-Seed-1.6, LLM=Qwen3-32B, T2IM=Qwen-Image, Resolution=1024x1024, Max Iterations=52025.11 | 50 | |
| VisualPunMLLM=Grok-4, LLM=Llama-4-Maverick, T2IM=Qwen-Image, Resolution=1024x1024, Max Iterations=52025.11 | 49.6 | |
| VisualPunMLLM=Doubao-Seed-1.6, LLM=Doubao-Seed-1.6, T2IM=Qwen-Image, Resolution=1024x1024, Max Iterations=52025.11 | 49.5 | |
| VisualPunMLLM=Qwen2.5-VL-32B, LLM=Gemini-2.5-flash, T2IM=Qwen-Image, Resolution=1024x1024, Max Iterations=52025.11 | 49.3 | |
| VisualPunMLLM=Gemma-3-27b, LLM=Doubao-Seed-1.6, T2IM=Qwen-Image, Resolution=1024x1024, Max Iterations=52025.11 | 49.2 | |
| VisualPunMLLM=GLM-4.5V, LLM=GPT-5, T2IM=Qwen-Image, Resolution=1024x1024, Max Iterations=52025.11 | 48.7 | |
| VisualPunMLLM=Doubao-Seed-1.6, LLM=GLM-4.5, T2IM=Qwen-Image, Resolution=1024x1024, Max Iterations=52025.11 | 48 | |
| VisualPunMLLM=Gemma-3-27b, LLM=GLM-4.5, T2IM=Qwen-Image, Resolution=1024x1024, Max Iterations=52025.11 | 47.9 | |
| VisualPunMLLM=Llama-4-Maverick, LLM=Claude-Sonnet-4, T2IM=Qwen-Image, Resolution=1024x1024, Max Iterations=52025.11 | 47.8 | |
| VisualPunMLLM=Qwen2.5-VL-32B, LLM=GPT-OSS-120b, T2IM=Qwen-Image, Resolution=1024x1024, Max Iterations=52025.11 | 47.6 | |
| VisualPunMLLM=Gemma-3-27b, LLM=Llama-4-Maverick, T2IM=Qwen-Image, Resolution=1024x1024, Max Iterations=52025.11 | 47.4 | |
| VisualPunMLLM=Doubao-Seed-1.6, LLM=Llama-4-Maverick, T2IM=Qwen-Image, Resolution=1024x1024, Max Iterations=52025.11 | 47.1 | |
| VisualPunMLLM=Qwen2.5-VL-32B, LLM=DeepSeek-V3.1, T2IM=Qwen-Image, Resolution=1024x1024, Max Iterations=52025.11 | 46.9 | |
| VisualPunMLLM=GLM-4.5V, LLM=Gemini-2.5-flash, T2IM=Qwen-Image, Resolution=1024x1024, Max Iterations=52025.11 | 46.8 | |
| VisualPunMLLM=Qwen2.5-VL-32B, LLM=Grok-4, T2IM=Qwen-Image, Resolution=1024x1024, Max Iterations=52025.11 | 46.5 | |
| VisualPunMLLM=Qwen2.5-VL-32B, LLM=Qwen3-32B, T2IM=Qwen-Image, Resolution=1024x1024, Max Iterations=52025.11 | 46 | |
| VisualPunMLLM=Llama-4-Maverick, LLM=GPT-5, T2IM=Qwen-Image, Resolution=1024x1024, Max Iterations=52025.11 | 45.9 | |
| VisualPunMLLM=GLM-4.5V, LLM=GPT-OSS-120b, T2IM=Qwen-Image, Resolution=1024x1024, Max Iterations=52025.11 | 45.2 | |
| VisualPunMLLM=Qwen2.5-VL-32B, LLM=Doubao-Seed-1.6, T2IM=Qwen-Image, Resolution=1024x1024, Max Iterations=52025.11 | 45.1 | |
| VisualPunMLLM=GLM-4.5V, LLM=DeepSeek-V3.1, T2IM=Qwen-Image, Resolution=1024x1024, Max Iterations=52025.11 | 44.5 | |
| VisualPunMLLM=GLM-4.5V, LLM=Grok-4, T2IM=Qwen-Image, Resolution=1024x1024, Max Iterations=52025.11 | 44.2 | |
| VisualPunMLLM=Llama-4-Maverick, LLM=Gemini-2.5-flash, T2IM=Qwen-Image, Resolution=1024x1024, Max Iterations=52025.11 | 44.1 | |
| VisualPunMLLM=Qwen2.5-VL-32B, LLM=GLM-4.5, T2IM=Qwen-Image, Resolution=1024x1024, Max Iterations=52025.11 | 44.1 | |
| VisualPunMLLM=GLM-4.5V, LLM=Qwen3-32B, T2IM=Qwen-Image, Resolution=1024x1024, Max Iterations=52025.11 | 43.7 | |
| VisualPunMLLM=Qwen2.5-VL-32B, LLM=Llama-4-Maverick, T2IM=Qwen-Image, Resolution=1024x1024, Max Iterations=52025.11 | 43.4 | |
| VisualPunMLLM=GLM-4.5V, LLM=Doubao-Seed-1.6, T2IM=Qwen-Image, Resolution=1024x1024, Max Iterations=52025.11 | 42.8 | |
| VisualPunMLLM=Llama-4-Maverick, LLM=GPT-OSS-120b, T2IM=Qwen-Image, Resolution=1024x1024, Max Iterations=52025.11 | 42.6 | |
| VisualPunMLLM=Llama-4-Maverick, LLM=DeepSeek-V3.1, T2IM=Qwen-Image, Resolution=1024x1024, Max Iterations=52025.11 | 42.1 | |
| VisualPunMLLM=GLM-4.5V, LLM=GLM-4.5, T2IM=Qwen-Image, Resolution=1024x1024, Max Iterations=52025.11 | 41.9 | |
| VisualPunMLLM=Llama-4-Maverick, LLM=Grok-4, T2IM=Qwen-Image, Resolution=1024x1024, Max Iterations=52025.11 | 41.6 | |
| VisualPunMLLM=Llama-4-Maverick, LLM=Qwen3-32B, T2IM=Qwen-Image, Resolution=1024x1024, Max Iterations=52025.11 | 41.2 | |
| VisualPunMLLM=GLM-4.5V, LLM=Llama-4-Maverick, T2IM=Qwen-Image, Resolution=1024x1024, Max Iterations=52025.11 | 41.2 | |
| VisualPunMLLM=Llama-4-Maverick, LLM=Doubao-Seed-1.6, T2IM=Qwen-Image, Resolution=1024x1024, Max Iterations=52025.11 | 40.3 | |
| VisualPunMLLM=Llama-4-Maverick, LLM=GLM-4.5, T2IM=Qwen-Image, Resolution=1024x1024, Max Iterations=52025.11 | 39.5 | |
| VisualPunMLLM=Llama-4-Maverick, LLM=Llama-4-Maverick, T2IM=Qwen-Image, Resolution=1024x1024, Max Iterations=52025.11 | 38.8 | |
| VisualPunMLLM=Mistral-Small-3.2, LLM=Claude-Sonnet-4, T2IM=Qwen-Image, Resolution=1024x1024, Max Iterations=52025.11 | 29.4 | |
| VisualPunMLLM=Mistral-Small-3.2, LLM=GPT-5, T2IM=Qwen-Image, Resolution=1024x1024, Max Iterations=52025.11 | 28.2 | |
| VisualPunMLLM=Mistral-Small-3.2, LLM=Gemini-2.5-flash, T2IM=Qwen-Image, Resolution=1024x1024, Max Iterations=52025.11 | 27.1 | |
| VisualPunMLLM=Mistral-Small-3.2, LLM=GPT-OSS-120b, T2IM=Qwen-Image, Resolution=1024x1024, Max Iterations=52025.11 | 26.2 | |
| VisualPunMLLM=Mistral-Small-3.2, LLM=DeepSeek-V3.1, T2IM=Qwen-Image, Resolution=1024x1024, Max Iterations=52025.11 | 25.9 | |
| VisualPunMLLM=Mistral-Small-3.2, LLM=Grok-4, T2IM=Qwen-Image, Resolution=1024x1024, Max Iterations=52025.11 | 25.6 | |
| VisualPunMLLM=Mistral-Small-3.2, LLM=Qwen3-32B, T2IM=Qwen-Image, Resolution=1024x1024, Max Iterations=52025.11 | 25.3 | |
| VisualPunMLLM=Mistral-Small-3.2, LLM=Doubao-Seed-1.6, T2IM=Qwen-Image, Resolution=1024x1024, Max Iterations=52025.11 | 24.8 | |
| VisualPunMLLM=Mistral-Small-3.2, LLM=GLM-4.5, T2IM=Qwen-Image, Resolution=1024x1024, Max Iterations=52025.11 | 24.3 | |
| VisualPunMLLM=Mistral-Small-3.2, LLM=Llama-4-Maverick, T2IM=Qwen-Image, Resolution=1024x1024, Max Iterations=52025.11 | 24.1 |