Zero-shot Language Modeling on LM Evaluation Harness benchmark leaderboard · SOTA2 Research

Zero-shot Language Modeling on LM Evaluation Harness

80.66WGLO-BCQ

Evaluation Results

Method	Links
LO-BCQ2025.02		80.66	49.28	84.93	81.34	65.18	72.28	0.12
BF162025.02		79.95	48.8	85.23	81.56	65.27	72.16	—
LO-BCQ2025.02		78.77	49	82.82	81.45	64.21	71.25	0.91
LO-BCQ2025.02		78.37	49.28	84.03	81.45	64.76	71.58	0.58
VSQ2025.02		77.27	47.85	82.29	79.82	61.4	69.73	2.43
MX42025.02		76.4	48.04	82.41	80.58	63.24	70.13	2.03
MXFP42025.02		76.32	47.75	83.06	80.58	63.24	70.19	1.97
QuaRot2025.02		76.24	—	—	82.43	—	—	—
BF162025.02		70.64	40.67	76.54	79.16	57.11	64.82	—
LO-BCQ2025.02		70.17	39.43	77.09	78.62	56.6	64.38	0.44
LO-BCQ2025.02		69.77	42.58	77.43	77.09	56.51	64.68	0.97
VSQ2025.02		69.61	40.57	65.81	77.2	54.82	61.6	3.22
BF162025.02		69.38	44.4	79.29	78.07	57.1	65.65	—
LO-BCQ2025.02		69.3	39.62	75.35	78.89	56.64	63.96	0.86
LO-BCQ2025.02		69.14	40.48	75.41	78.24	56.06	63.87	0.95
LO-BCQ2025.02		68.9	42.49	77.58	77.09	55.93	64.4	1.25
LO-BCQ2025.02		68.9	43.73	77.86	77.86	56.52	64.97	0.68
MX42025.02		67.96	39.04	72.26	77.86	54.77	62.38	2.44
BF162025.02		67.88	41.34	68.32	78.78	54.16	62.1	—
LO-BCQ2025.02		67.72	39.43	69.45	77.75	53.71	61.61	0.49
MXFP42025.02		67.48	41.34	74	77.53	54.22	62.91	2.74
MXFP42025.02		67.01	39.71	65.35	76.12	50.22	59.68	2.42
LO-BCQ2025.02		67	39.62	69.3	77.37	53.51	61.36	0.74
VSQ2025.02		66.93	40.86	63.91	76.28	51.38	59.87	2.23
LO-BCQ2025.02		66.85	40.48	69.2	77.31	53.06	61.38	0.72
MX42025.02		66.22	41.43	73.98	77.04	55.19	62.77	2.88
MX42025.02		65.11	38.28	66.27	75.63	50.77	59.21	2.89
MXFP42025.02		64.17	39.14	69.61	75.68	47.6	59.24	5.58
QuaRot2025.02		63.77	—	—	76.77	—	—	—
VSQ2025.02		55.49	31.39	65.75	67.3	43.51	52.69	12.96