Zero-shot Classification on MMLU, OBQA, ARC, PIQA, HellaSwag, WinoGrande benchmark leaderboard · SOTA2 Research

Zero-shot Classification on MMLU, OBQA, ARC, PIQA, HellaSwag, WinoGrande

37.1MMLU (5-shot)LLAMA-7B

Evaluation Results

Method	Links
LLAMA-7B2023.05		37.1	42.4	67.45	67.01	67.45	78.35	72.99	65.34
LoRAPrune2023.05		36.81	38.6	70.2	66.77	41.89	77.48	68.64	62.7
LoRAPrune-8bit2023.05		36.45	38.1	70.25	65.93	41.43	77.1	68.9	60.29
LLM-Pruner2023.05		33.67	38.8	70.62	65.82	40.7	77.37	66.6	62.36
LoRAPrune2023.05		33.6	36.2	69.61	62.75	41.21	77.48	66.68	58.98
LoRAPrune-8bit2023.05		33.36	36.4	69.48	62.31	41.93	77.4	65.91	58.9
Compresso2023.05		31.9	36.4	68.64	67.8	37.97	75.46	53.44	59.82
LLM-Pruner2023.05		30.67	34.86	66.2	63.85	40.55	75.6	65.12	57.7
WANDA2023.05		28.35	35.26	68.96	64.01	38.46	74.8	52.63	58.68
WANDA2023.05		27.9	34.9	65.06	61.16	39.44	74.38	60.84	55.96
Magnitude2023.05		27.84	35.8	65.36	61.33	38.74	74.87	63.9	56.67
LoRAPrune2023.05		27.84	35.8	60.38	56.97	40.12	75.39	60.21	54.81
LoRAPrune-8bit2023.05		27.71	35.3	60.54	56.13	40.58	74.89	59.86	54.55
Compresso2023.05		27.68	29.8	66.23	64.8	37.2	75.63	49.16	53.79
Magnitude2023.05		26.38	33.67	65.58	60.79	37.47	73.15	60.35	55.16
LLM-Pruner2023.05		25.6	33.12	55.36	56.12	34.98	73.25	58.6	51.9
WANDA2023.05		24.15	28.78	52.68	55.98	34.2	70.38	54.12	49.35
Magnitude2023.05		21.84	30.26	53.61	55.86	36.98	67.1	53.1	49.48