Long-horizon Research Task Reproduction on PaperBench Code (dev) benchmark leaderboard · SOTA2 Research

Long-horizon Research Task Reproduction on PaperBench Code (dev)

72.22FRE ScoreClaude 4.5 Sonnet

Evaluation Results

Method	Links
Claude 4.5 Sonnet2026.02		72.22	86.76	70.42	64.28	57.31	92.75	36.7	82.74	63.03	81.85	42.01	83.35	79.37	54.89	51.81	83.38	68.44	75.65	90.89	57.14	69.75
DeepSeek-V3.22026.02		70.7	68.7	40.3	62.79	60.58	59.29	35.89	23.02	63.29	4.07	29.13	61.01	66.06	15.51	4.95	82.42	34.07	72.85	44.71	42.93	47.11
Kimi K2 Thinking2026.02		69.55	58.52	35.06	65.64	28.1	57.2	35.05	71.01	61.29	23.23	62.57	52.09	59.78	32.1	21.35	79.76	28.93	74.65	83.08	27.26	51.31
Grok 42026.02		62.34	56.32	24.64	15.91	18.82	52	37.34	58.54	79.61	68.15	42.21	25.92	38.02	27.97	34.66	46.07	55.48	67.41	76.36	56.23	47.2
KLong2026.02		61.04	75.65	51.25	80.09	70.14	80.13	42.71	56.93	69.61	56.14	54.62	71.41	63.63	55.27	68.4	68.08	47	82.12	62.02	35.61	62.59
GPT-5 Thinking2026.02		33.23	68.7	56.85	48.94	39.07	73.44	49.6	47.77	51.5	57.25	40.43	75.54	28.5	63.71	48.81	64.25	32.9	69.93	62.61	33.15	52.31
Qwen3-Thinking2026.02		15.83	21.67	28.87	44.15	1.5	46.97	23.96	11.98	34.13	18.29	34.38	74.04	19.42	20.71	2.56	51.55	14.34	21.67	68.44	20.05	28.72