Zero-shot Evaluation on MobileLLM Evaluation Suite

69.19ARC-eEDGERAZOR

Evaluation results

Method
EDGERAZOR2026.04	69.19	36.26	51.91	62.26	70.4	56.2	40.74	37.4	37.96	57.41	25	53	41.94
EDGERAZOR2026.04	69.11	35.84	51.82	62.6	70.35	56.2	40.58	37.4	37.9	57.21	24.66	45	41.86
EDGERAZOR2026.04	65.99	32.68	45.99	62.11	68.55	56.51	40.07	35.2	39.05	56.51	24.41	99	40.62
EDGERAZOR2026.04	65.87	32.68	45.98	61.71	68.82	56.27	40.02	35	38.97	56.53	24.27	76	40.53
MobileLLM-350M2026.04	64.94	35.49	52.87	58.96	70.84	56.35	40.79	40.2	37.44	53.98	23.52	0	41.18
ParetoQ2026.04	64.23	38.14	53.13	58.32	71.55	56.2	40.33	38	37.04	50.73	24.78	8	40.96
EfficientQAT2026.04	63.68	35.67	51.73	58.47	70.73	56.75	40.74	38.4	37.11	54.07	24.16	0	40.89
ParetoQ2026.04	62.75	33.28	51.24	60.92	70.95	56.75	39.82	39	37	46.39	25.02	0	40.24
FlatQuant2026.04	61.95	34.73	51.72	58.93	70.84	53.75	40.33	39	38.01	52.24	23.56	15	40.4
EfficientQAT2026.04	61.53	33.11	49.45	60.89	69.04	53.75	39.66	36.8	37.66	45.35	22.24	0	39.27
EDGERAZOR2026.04	61.36	29.18	40.86	58.23	66.92	55.49	39.56	33.2	40.95	56.13	24.97	38	39.02
EDGERAZOR2026.04	61.2	28.75	40.76	58.23	66.59	55.01	39.51	33	40.98	56.22	25.03	53	38.91
FlatQuant2026.04	61.07	31.91	48.43	55.44	68.39	53.12	39.51	35.2	39.24	45.23	24.74	8	38.64
EDGERAZOR2026.04	58.67	26.19	38.92	58.04	65.23	53.83	39.25	32	42.03	56.33	24.19	83	38.12
EDGERAZOR2026.04	58.63	26.19	38.95	58.07	65.29	53.04	39.3	32.2	41.97	56.26	24.12	53	38.04
ParetoQ2026.04	57.66	32.59	46.95	63.03	69.31	56.67	40.43	35.2	36.25	43.4	24.88	45	38.99
ParetoQ2026.04	56.1	29.95	43.68	61.62	67.3	54.3	39.3	36.4	38.82	43.34	23.05	15	38
EfficientQAT2026.04	49.92	27.05	39.29	61.77	63.49	50.91	37.56	29.6	42.5	46.05	22.93	0	36.24
FlatQuant2026.04	31.44	22.27	27.19	44.71	51.74	47.59	34.6	25.4	49.22	43.53	23.03	0	30.82
AutoRound2026.04	26.39	29.69	25.79	61.71	50.33	48.78	33.62	27.6	48.14	43.6	26.9	0	32.5
QTIP2026.04	26.35	28.24	26.4	37.83	49.08	50.59	35.16	24.6	49.21	56.67	22.97	0	31.32
AutoRound2026.04	25.88	29.27	25.86	60	49.95	51.62	33.16	26.2	47.95	46.53	27.12	0	32.58
AutoRound2026.04	25.72	29.27	25.81	61.9	50.33	49.8	33.78	27	47.78	43.59	27	0	32.46