Binary Question Answering on ACaM Synthetic Videos (test) benchmark leaderboard · SOTA2 Research

Binary Question Answering on ACaM Synthetic Videos (test)

99Accuracy (Static)Human Performance

Evaluation Results

Method	Links
Human Performance2026.07		99	99.64	98.58	98.59	100	100	99.15	99.15
Our SFT Qwen3-VL-4B2026.07		88.42	82.6	74.44	81.42	70.23	73.95	78.28	78.62
Mega-SaM2026.07		86	80.2	67.1	—	—	—	—	—
GPT-52026.07		78.65	70.75	64.05	79.66	69.88	82.35	70.22	70.2
Qwen3-VL-4B2026.07		76.03	72.26	59.76	65.38	59.18	72.73	66.58	65.44
ShotVL-7B2026.07		75.23	76.82	68.08	72.57	56.25	83.19	72.29	72.73
Qwen3-VL-32B2026.07		75	81.3	65.34	70.83	72.16	78.57	73.31	72.82
Gemini-3-Pro2026.07		75	63.4	66.67	61.18	80	86.96	68.26	65.42
CamReasoner-7B2026.07		74.07	78.64	70.15	76.8	77.19	94.23	75.98	76.24
InternVL3.5-8B2026.07		72.92	59.68	45.09	64	25.4	69.14	54.79	52.17
Our SFT Qwen3-VL-8B2026.07		70.89	87.18	78.73	79.63	68.75	79.25	80.56	80.69
Gemini-3.1-Pro2026.07		70.83	66.97	62.96	59.09	71.91	87.91	67.09	64.76
ViPE2026.07		66.7	58.9	76.6	—	—	—	—	—
Qwen3-VL-8B2026.07		57.53	77.48	62.99	72.55	61.39	84.78	69.88	69.65
CameraModel-7B2026.07		57.43	55.21	56.99	52.73	70.97	47.06	56.53	56.21
Random Guess2026.07		50	50	50	50	50	50	50	50
LLaVa-OV-7B2026.07		35.56	49.67	31.3	27.27	59.18	72.16	43.81	39.71
InternVL3.5-14B2026.07		24.56	58.58	60.31	69.16	70.09	77.55	61.01	57.41
Spatial-MLLM2026.07		23.88	65.63	55.09	64.97	64.38	70.42	60.8	57.06
VLM-3R-7B2026.07		21.21	35.23	36.03	26.37	19.35	70.73	35.73	31.5
InternVideo2.5-8B2026.07		18.18	67.5	53.22	68.22	67.65	83.93	61.68	58.06
G2VLM-2B2026.07		11.11	8.58	9.55	0	7.41	30	9.76	8.99
Qwen2.5-VL-7B2026.07		10.91	69.96	66.46	67.23	61.95	82.57	66.58	65.29