Long-context Multimodal Evaluation on MileBench (test) benchmark leaderboard · SOTA2 Research

Long-context Multimodal Evaluation on MileBench (test)

25.34TN ScoreFull Cache

Evaluation Results

Method	Links
Full Cache2025.06		25.34	9.01	40.5	15.7	18.56	15.98	55.5	74.5	52	34.12
MadaKV2025.06		23.43	9.41	38.5	15.62	17.76	16.76	55	74.5	51.5	33.61
Full Cache2025.06		11.25	29.45	44.5	28.36	37.53	42.46	62.5	63	61	63.34
MadaKV2025.06		10.73	29.39	43.5	27.3	37.95	40.57	62.5	62	62.5	62.74
LOOK-M2025.06		10	8.77	37.5	15.46	15.25	14.12	53	74	50.5	30.96
Full Cache2025.06		9.68	7.98	33.5	16.32	15.18	16.62	46	63.5	48.5	28.59
MadaKV2025.06		9.38	6.97	31	15.85	15.06	16.76	47	64	48	28.22
SnapKV2025.06		7.92	8.52	35	15.53	16.84	15.37	53.5	74.5	51	30.91
StreamingLLM2025.06		7.81	2.63	28.5	14.05	1.72	6.55	49.8	68	50	25.45
H2O2025.06		7.81	8.7	34.5	15.21	15.03	15.06	53	74	51	30.48
SnapKV2025.06		4.55	29.25	41.5	26.54	37.41	39.41	61.5	60	60.5	60.11
LOOK-M2025.06		4.23	28.73	42	28.97	36.66	38.66	62	61.5	61.5	60.71
H2O2025.06		3.97	29.13	40.5	26.79	37.78	36.03	62.5	61.5	61	59.7
StreamingLLM2025.06		3.46	25.12	40	26.14	34.07	27.58	60	59	60.5	55.98
LOOK-M2025.06		3.34	6.51	29.5	15.79	13.96	14.12	45.5	63	46.5	26.47
SnapKV2025.06		3.27	6.03	29	14.82	14.4	15.37	45.5	64	45.5	26.43
StreamingLLM2025.06		3.12	3.59	26	11.77	3.73	10.44	42.5	43	44	20.91
H2O2025.06		2.5	5.51	28	15.73	14.86	14.07	44	63.5	44	25.8