Question Answering on COTEMPQA 1.0 (test) benchmark leaderboard · SOTA2 Research

Question Answering on COTEMPQA 1.0 (test)

97Equal AccuracyHUMAN

Evaluation Results

Method	Links
HUMAN2024.06		97	98.3	97.7	91.1	93.5	92.3	82	87	84.5	88	96.2	92.1	92.8
GPT-42024.06		91.1	94.3	92.7	55.3	63.5	59.4	44.3	55.8	50.1	23.4	66.5	45	54.7
GPT-3.5-Turbo2024.06		59.4	66.3	62.8	40.1	48.5	44.3	31.5	42.9	37.2	0.7	46.1	23.4	38.9
WIZARDMATH2024.06		36.7	46.8	41.8	23.6	33.7	28.6	25.5	37.1	31.3	0.4	32.9	16.6	30.1
WIZARDMATH2024.06		23.9	29	26.4	10.9	15.1	13	11.7	17.1	14.4	0	13.2	6.6	14.4
WIZARDCODER2024.06		19.5	26.3	22.9	15.2	22.4	18.8	15.9	23.9	19.9	0.9	25.9	13.4	19.2
CODELLAMA2024.06		16.1	46.5	31.3	9.8	27	18.4	8.1	28.4	18.3	4.4	40.3	22.4	20
GPT-3.5-Turbo2024.06		13.8	14.8	14.3	11.3	14.3	12.8	15	22.9	18.9	0	15.5	7.7	16.3
WIZARDMATH2024.06		12.4	16.5	14.4	9.2	15.2	12.2	11.6	20.5	16	0.4	22	11.2	14.8
LLAMA2024.06		11.9	41.7	26.8	10	32.5	21.2	9.4	33.5	21.4	5.2	42.5	23.8	22.2
GPT-42024.06		11.2	12.3	11.8	11.5	14	12.7	14.8	18.5	16.7	0	13.6	6.8	14.5
LLAMA2024.06		11.2	31.2	21.2	5.8	21.6	13.7	5	20.6	12.8	1.1	26.9	14	13.8
WIZARDCODER2024.06		9.2	21.1	15.1	4.7	14.8	9.8	6.3	15.9	11.1	0.5	20.4	10.5	11.2
WIZARDCODER2024.06		8.3	16.6	12.4	7	17.8	12.4	9.5	19.7	14.6	1.1	24.1	12.6	13.9
CODELLAMA2024.06		7.6	28.3	18	4.1	17	10.6	3.3	19.3	11.3	3.2	28.6	15.9	12.4
CODELLAMA2024.06		6.4	27.7	17	3.1	14.6	8.8	3.1	15.8	9.5	2	24.1	13	10.5
LLAMA2024.06		4.1	18.9	11.5	4.7	19.5	12.1	4.5	19.5	12	0.2	23.8	12	12