World Knowledge Reasoning on WISE random subset of 200 samples
94Cultural AccuracyGPT-4o
Evaluation Results
| Method | Links | |||||||
|---|---|---|---|---|---|---|---|---|
| GPT-4o2025.04 | 94 | 64 | 98 | 93 | 98 | 95 | 89 | |
| MetaQuery-LModel Size=L2025.04 | 56 | 57 | 62 | 48 | 63 | 42 | 55 | |
| MetaQuery-XLModel Size=XL2025.04 | 56 | 55 | 62 | 49 | 63 | 41 | 55 | |
| playground-v2.52025.04 | 49 | 58 | 55 | 43 | 48 | 33 | 49 | |
| FLUX.1-dev2025.04 | 48 | 58 | 62 | 42 | 51 | 35 | 50 | |
| PixArt-Alpha2025.04 | 45 | 50 | 48 | 49 | 56 | 34 | 47 | |
| SD-3.5-largeModel Size=large2025.04 | 44 | 50 | 58 | 44 | 52 | 31 | 46 | |
| MetaQuery-BModel Size=B2025.04 | 44 | 49 | 58 | 41 | 49 | 34 | 46 | |
| SD-XL2025.04 | 43 | 48 | 47 | 44 | 45 | 27 | 43 | |
| SD-v1-52025.04 | 34 | 35 | 32 | 28 | 29 | 21 | 32 | |
| Emu32025.04 | 34 | 45 | 48 | 41 | 45 | 27 | 39 | |
| Janus-Pro-7BModel Size=7B2025.04 | 30 | 37 | 49 | 36 | 42 | 26 | 35 | |
| show-o-512Resolution=5122025.04 | 28 | 40 | 48 | 30 | 46 | 30 | 35 | |
| vila-u-7b-256Model Size=7b, Resolution=2562025.04 | 26 | 33 | 37 | 35 | 39 | 23 | 31 | |
| Janus-Pro-1BModel Size=1B2025.04 | 20 | 28 | 45 | 24 | 32 | 16 | 26 | |
| Janus-1.3BModel Size=1.3B2025.04 | 16 | 26 | 35 | 28 | 30 | 14 | 23 | |
| JanusFlow-1.3BModel Size=1.3B2025.04 | 13 | 26 | 28 | 20 | 19 | 11 | 18 |