Task-solving Performance on BIG-Bench Hard (test) benchmark leaderboard · SOTA2 Research

Task-solving Performance on BIG-Bench Hard (test)

68.3Causal JudgementBreeder

Updated 3mo ago

Evaluation Results

Method	Links
Breeder2025.10		68.3	—	89.8	74.5	—	74.6	68.4	93.2	77.8	72.4	—	—	—	90.8	—	92.7	—	—	69.8	90.3	—	—	85.9	60	92.7	96.3	—	—
OPRO2025.10		68.2	—	91	73.4	—	72.8	56.9	93.2	77.4	71.8	—	—	—	88.2	—	91.9	—	—	68.7	88.8	—	—	83.3	66.2	92.4	93.8	—	—
APE2025.10		68	—	89.2	73	—	74.7	67	94	82.2	72.1	—	—	—	89.9	—	92.5	—	—	68.6	87.5	—	—	80.3	60	91.1	94.8	—	—
PDO2025.10		68	—	91.5	74	—	75.4	71.2	94.8	80.9	73.3	—	—	—	89	—	93.3	—	—	69.5	87.4	—	—	84.7	66.2	94.6	97.9	—	—
EvoPrompt(DE)-OPTS(TS)2025.03		45.5	82.5	20.17	42.5	7.5	43.5	35.83	60.5	37.17	13	78.83	54.33	52.5	84.67	85.83	40.97	49.5	67.67	51.17	64.06	78.45	66	69.33	49.5	81.67	95	45.33	55.67
EvoPrompt(DE)-OPTS(APET)2025.03		42.34	79.83	19.17	38.33	6.5	44.83	33	70	29	10.17	70.17	49.5	52.17	80.17	85.67	28.47	50	68.67	46.5	58.59	61.62	65.83	66.17	52.5	77.5	95	45.83	52.87
EvoPrompt(DE)2025.03		40.39	74.5	17.17	30	6.67	40.67	36	54.67	14.67	5.83	45.83	49.5	53.5	83.17	85.83	22.57	53	27.17	46.17	55.47	61.45	65.83	66.5	52	78.67	95	45.5	48.43
EvoPrompt(DE)-OPTS(US)2025.03		40.15	84	18.67	47.33	6.17	42.5	33	59.5	24.67	13.17	71.83	48	50.33	81.67	85	41.67	45.67	64.83	51.17	65.1	76.94	60.33	68.17	54.33	79.17	95	46.17	53.87
Manual2025.03		2.19	54	14	19.5	6.5	29.5	16.5	53	12	5.5	44	40.5	53.5	84.5	87.5	26.04	21	18.5	12.5	24.22	51.52	57	67.5	47.5	80.5	93.5	44.5	39.52
APET2025.03		0	67.5	3	22.5	0	0	24.5	3.5	3.5	3	20.5	0	52	38.5	27.5	8.33	6.5	65.5	6.5	0	0	6	18	19.5	80.5	42	41	20.73