Spatial Reasoning on Escher-Bench 1.0 (test)

56.17Object Permanence & Occlusion TrackingGemini-2.5-pro

Evaluation results

Method
Gemini-2.5-pro2026.01	56.17	57.29	69.53	54.2	73.11	54.39	57.26	58.37	53.64	64.53	59.6	51.57	60.74
Gemini-2.5-flash2026.01	53.13	51.71	65.01	49.22	65.88	48.91	52.19	52.93	49.66	56.02	48.55	45.51	58.42
GPT52026.01	52.24	53.28	64.66	50.41	67.99	50.58	53.36	54.28	49.88	60.01	55.43	47.96	56.49
Escher-4B-Instruct2026.01	49.72	48.59	61.63	47.33	52.61	47.71	49.49	49.73	48.71	51.82	41.87	47	52.41
Qwen3-VL-32B-Thinking2026.01	49.26	49.8	58.76	49.34	60.19	46.04	49.58	49.7	49.18	57.68	48.55	43.3	53.87
Qwen3-VL-30B-A3B-Instruct2026.01	48.53	44.46	54.38	43.08	56.87	43.7	45.73	46.35	43.75	47.79	37.86	39.04	53.05
Escher-4B-Thinking2026.01	48.07	47.55	61.33	47.32	53.08	46.38	48.54	48.86	47.49	49.72	43.65	45.21	52.26
Escher-7B-Instruct2026.01	47.61	48.88	62.08	48.89	53.55	45.31	48.89	49.17	47.97	51.05	41.2	46.88	51.38
Escher-8B-Instruct2026.01	46.96	49.8	63.6	48.31	56.87	47.5	49.85	50.88	46.55	52.6	48.11	46.08	53.17
Qwen3-VL-8B-Instruct2026.01	46.69	43.98	55.89	43.08	58.29	42.21	45.06	45.74	42.86	47.79	38.31	37.82	52.6
ISI-8B-Instruct2026.01	46.59	43.01	56.5	43.46	50.24	43.64	45.06	46.06	41.86	50.61	26.5	38.92	52.32
Escher-8B-Thinking2026.01	46.04	48.67	62.84	48.64	60.19	45.22	48.79	49.12	47.71	50.28	44.99	46.08	51.66
Qwen3-VL-4B-Instruct2026.01	44.57	43.74	48.49	43	52.61	43.08	44.18	44.9	41.86	40.88	35.63	38.74	51.78
qwen3-vl-plus2026.01	44.38	43.21	55.52	44.61	57.82	42.38	44.75	45.83	41.28	44.86	44.77	38.63	50.94
MiniCPM-V-42026.01	43.19	38.67	46.53	37.94	32.23	38.54	39.62	39.81	39.06	35.25	22.27	36.65	46.22
Qwen3-VL-4B-Thinking2026.01	43.09	43.32	49.85	43.62	54.5	44.63	44.55	46.01	39.85	52.15	35.19	39.04	49.35
LLaVA-OneVision-1.5-4B-Instruct2026.01	42.82	43.35	48.64	41.52	50.71	43.64	43.73	44.27	41.96	44.64	36.3	38.56	49.74
LLaVA-OneVision-1.5-8B-Instruct2026.01	41.71	44.87	48.94	42.59	49.29	44.67	44.5	45.22	42.17	42.87	36.75	39.34	51.26
ERNIE-4.5-VL-28B-A3B-Thinking2026.01	41.34	40.17	51.66	40.28	57.35	41.22	42.58	43.11	41.08	49.72	35.19	37.85	49.01
Qwen3-VL-8B-Thinking2026.01	41.25	44.26	51.21	41.02	58.29	44.45	44.36	45.7	40.06	51.82	41.43	38.62	48.56
Qwen2.5-VL-7B-Instruct2026.01	39.32	35.73	45.17	38.02	38.39	33.75	36.84	36.69	37.32	26.08	12.92	32.39	47.59
Qwen2.5-VL-3B-Instruct2026.01	39.13	38.98	47.13	41.89	44.55	40.15	40.61	41.12	38.96	36.41	11.36	35.46	50.99
Qwen3-VL-2B-Instruct2026.01	37.75	37.53	44.11	38.35	41.23	38.63	38.65	39.62	35.53	31.71	7.8	34.27	49.23
Spatial-MLLM2026.01	10.6	15.2	26.13	22.8	11.37	12.81	15.82	15.6	16.55	16.02	1.11	33.28	0
ViCA2-7B2026.01	9.67	15.24	27.79	23.97	14.69	13.92	16.45	15.96	18.03	17.57	2	34.21	0.03
LLaVA-NeXT-Video-7B2026.01	6.26	10.28	8.91	16.05	2.84	13.66	11.3	10.63	13.44	1.1	6.46	25.8	0