List your company
ResearchFormerly WizWand

High-level planning on VLABench vlm_evaluation 1.0 (test)

85.8AUCOracle
30.669644.982359.29573.6077Jun 10, 2026
Updated 4mo ago

Evaluation results

MethodLinks
OraclePair=RynnBrain-Plan / GLM 4.1 T, Category=Heterogeneous Model Pairs, Sample Size=N=9002026.0685.8
OraclePair=Qwen3-VL 8B I / T, Category=Homogeneous Open-Weight Pairs, Sample Size=20% subsample2026.0681.92
OraclePair=GLM-4.6 I / T, Category=Homogeneous Open-Weight Pairs, Sample Size=N=9002026.0681.77
OraclePair=Cosmos-R2 / Qwen3-VL 8B T, Category=Heterogeneous Model Pairs, Sample Size=20% subsample2026.0679.97
OraclePair=GLM-4.6 I / 4.1 T, Category=Homogeneous Open-Weight Pairs, Sample Size=N=9002026.0676.99
DIRECTPair=Qwen3-VL 8B I / T, Category=Homogeneous Open-Weight Pairs, Sample Size=20% subsample2026.0659.57
DIRECTPair=Cosmos-R2 / Qwen3-VL 8B T, Category=Heterogeneous Model Pairs, Sample Size=20% subsample2026.0657.32
DIRECTPair=GLM-4.6 I / 4.1 T, Category=Homogeneous Open-Weight Pairs, Sample Size=N=9002026.0656.34
DIRECTPair=GLM-4.6 I / T, Category=Homogeneous Open-Weight Pairs, Sample Size=N=9002026.0655.47
RandomPair=GLM-4.6 I / 4.1 T, Category=Homogeneous Open-Weight Pairs, Sample Size=N=9002026.0648.22
Strongest-per-categoryPair=Qwen3-VL 8B I / T, Category=Homogeneous Open-Weight Pairs, Sample Size=20% subsample2026.0648.22
Strongest-per-categoryPair=GLM-4.6 I / 4.1 T, Category=Homogeneous Open-Weight Pairs, Sample Size=N=9002026.0647.95
Strongest-per-categoryPair=Cosmos-R2 / Qwen3-VL 8B T, Category=Heterogeneous Model Pairs, Sample Size=20% subsample2026.0647.94
Strongest-per-categoryPair=GLM-4.6 I / T, Category=Homogeneous Open-Weight Pairs, Sample Size=N=9002026.0646.6
DIRECTPair=RynnBrain-Plan / GLM 4.1 T, Category=Heterogeneous Model Pairs, Sample Size=N=9002026.0646.32
OODPair=Cosmos-R2 / Qwen3-VL 8B T, Category=Heterogeneous Model Pairs, Sample Size=20% subsample2026.0645.75
RandomPair=Qwen3-VL 8B I / T, Category=Homogeneous Open-Weight Pairs, Sample Size=20% subsample2026.0645.64
RandomPair=GLM-4.6 I / T, Category=Homogeneous Open-Weight Pairs, Sample Size=N=9002026.0644.57
OODPair=Qwen3-VL 8B I / T, Category=Homogeneous Open-Weight Pairs, Sample Size=20% subsample2026.0644.27
RandomPair=Cosmos-R2 / Qwen3-VL 8B T, Category=Heterogeneous Model Pairs, Sample Size=20% subsample2026.0643.62
OODPair=GLM-4.6 I / 4.1 T, Category=Homogeneous Open-Weight Pairs, Sample Size=N=9002026.0642.07
OODPair=GLM-4.6 I / T, Category=Homogeneous Open-Weight Pairs, Sample Size=N=9002026.0640.95
Strongest-per-categoryPair=RynnBrain-Plan / GLM 4.1 T, Category=Heterogeneous Model Pairs, Sample Size=N=9002026.0636.05
RandomPair=RynnBrain-Plan / GLM 4.1 T, Category=Heterogeneous Model Pairs, Sample Size=N=9002026.0632.95
OODPair=RynnBrain-Plan / GLM 4.1 T, Category=Heterogeneous Model Pairs, Sample Size=N=9002026.0632.79