Language Modeling and Reasoning Evaluation on Open LLM Leaderboard benchmark leaderboard · SOTA2 Research

Language Modeling and Reasoning Evaluation on Open LLM Leaderboard

82.8ARCDPO

Evaluation Results

Method	Links
DPO2024.06		82.8	50.6	74.3	52	65.8	63.1	—
KTO2024.06		82.7	48.7	74.9	53.9	65.4	63	—
AOT paired2024.06		82.5	50.8	74.4	53.1	66.1	62.9	—
AOT unpaired2024.06		82.5	51.1	74.4	51.8	66.2	62.8	—
IPO2024.06		82.4	46.5	74	52.3	65.1	63	—
Merlinite-7B2024.06		81.6	42	73.9	45.2	63.2	62.6	—
FT2024.02		73.34	47.07	74.03	16.22	79.95	46.55	56.19
LoRA2024.02		72.07	44.84	75.85	15.01	78.6	44.42	55.13
RED2024.02		72.04	47.59	72.85	11.9	77.86	42.27	54.09
FP162024.04		59.4	37.4	76.6	22.5	82.1	55.7	55.6
FP162024.04		59.4	37.4	76.6	22.5	82.1	55.7	55.6
FP162024.04		59.4	37.4	76.6	22.5	82.1	55.7	55.6
CherryQ2024.04		58.6	38.8	77	21	82	54.6	55.3
QAT2024.04		57.9	38.9	75.7	19.6	81.9	54.2	54.7
GPTQ2024.04		57.4	36.1	76.8	20.4	81.5	54.6	54.5
CherryQ2024.04		57.3	38	76.2	18.4	81.1	53.5	54.1
CherryQ2024.04		56.7	38.9	75.4	17.8	81	52.5	53.7
GPTQ2024.04		56.5	36	74.4	16.4	79.2	52.4	52.5
QAT2024.04		55.5	39	75.1	16.8	80.7	52.9	53.3
QAT2024.04		55.3	38.8	75.5	16	80.7	51.9	53
GPTQ2024.04		54.1	34.9	75.4	15.6	79.1	50.3	51.6
FP162024.04		53.2	38.8	74	14.5	78.6	46.7	51
FP162024.04		53.2	38.8	74	14.5	78.6	46.7	51
FP162024.04		53.2	38.8	74	14.5	78.6	46.7	51
QAT2024.04		52	39	72.2	10.6	77.5	43.7	49.2
GPTQ2024.04		52	39.1	72.9	11.1	77.6	43.8	49.4
CherryQ2024.04		51.5	39.5	73.5	12.9	77.8	44.4	49.9
CherryQ2024.04		50.6	38.6	71.8	10.4	77	43.9	48.7
CherryQ2024.04		49.7	38.1	72.4	8.8	76.3	41.6	47.8
QAT2024.04		49.2	37.3	71.6	7.3	75.5	40.6	46.9
GPTQ2024.04		48.6	38.8	71.7	8.1	73.9	39.4	46.8
GPTQ2024.04		48.6	39.1	70.8	5.4	72.9	38.2	45.8
QAT2024.04		48.2	37.7	70.8	6.7	75.4	39	46.3