Complex engineering solution design on SolutionBench 1.0 (test) benchmark leaderboard · SOTA2 Research

Complex engineering solution design on SolutionBench 1.0 (test)

66.4Environmental Score (AS)SolutionRAG

Evaluation Results

Method	Links
SolutionRAG2025.02		66.4	67.9	59.7	50.5	64.1	58.5	69.9	72.7	68.8	69	67.9	68	66	60.7	66.9	65.2
Naïve-RAG2025.02		64.8	62.2	57.2	40.1	62.7	54.9	67.7	65.4	67.4	66.8	66.2	63.3	66	57.5	65.7	63
Self-RAG2025.02		64.2	63.6	56.1	41.6	62.9	56.5	68.8	69.9	67.6	66.9	66.7	65.9	64.8	58.6	65.1	61.1
SolutionRAG2025.02		63.5	66.5	57.3	46.2	63.1	57.4	60.8	68.4	60.9	63.7	66.2	67.2	65.6	59.9	64.2	63.9
SolutionRAG2025.02		62.8	64.7	55.6	47.3	61.5	55.7	63.2	68.3	62.6	64.8	67.5	67.3	64.4	59.1	65.2	64.7
Rerank-RAG2025.02		62.7	60.7	53.4	38.4	60	49.7	65.6	65.2	66.1	63.4	66.4	62.8	64.1	55.4	64	59.7
o1-2024-12-172025.02		60.5	48.3	51.9	37.5	57.3	44.7	57.8	47.6	63.5	52.3	61.2	52	59.9	50.4	62.9	52.2
GenGround2025.02		54.8	46.1	53	33.3	54.7	37.2	55.7	46	58.3	50.7	60.1	50.7	60.4	48.9	59.8	52.7
QwQ-32B-Preview2025.02		54.3	38.7	48	27.9	47.2	29.3	47.4	31.9	52.2	35.9	51.3	35.6	49.2	33	53.4	37
RQ-RAG2025.02		53.5	44.4	48.9	28.7	53.8	38.8	55	46.1	57.9	44.6	56.3	46.9	54.3	39.8	57.2	45.2
GLM-Zero-Preview2025.02		47	30.6	43.2	22.2	45.2	27	42.3	25.7	45.1	31.7	47.7	32.4	47.3	30.8	51.4	36.6