Interactive Science Simulation on ScienceWorld v1.0 (test) benchmark leaderboard · SOTA2 Research

Interactive Science Simulation on ScienceWorld v1.0 (test)

97.04Task 1-1 (L) ScoreSwiftSage

Updated 5mo ago

Evaluation Results

Method	Links
SwiftSage2023.05		97.04	87.04	72.78	100	99.17	88.17	95.73	88.67	55.33	71.9	77.86	100	100	91.67	100	74.59	93.93	49.4	100	91.48	95	85	93.33	89	68.5	75	70	60	92.3	77.6	92.22	77.79	83	84.68
SayCan2023.05		33.06	10.39	3.88	0.37	26.37	8.03	17.41	52.14	22.5	99.56	47.76	22.87	58.18	20.87	31.43	9.92	13.93	47.81	39.26	19.72	80	67.5	50	20.91	16	21.94	32.26	13.67	67.53	59.45	43.83	36.58	23.65	33.82
Reflexion2023.05		4.22	10.61	7.78	0.92	5.92	28.59	22.37	100	17.45	72.54	70.22	64.93	87.27	16.42	100	7.33	13	70.35	70.67	15.77	100	84.37	83	2.58	8	50.63	100	70.62	50.9	23.69	71.47	35.43	30.17	45.34
DRRN2023.05		3.52	3.52	0	0	6.56	5.5	6	12	9	9.05	9.52	15	45	21.67	19.17	8	14.29	15.77	26.67	10.37	50	50	33.33	21	8	10	10	10	16.8	17	28.08	10.85	8.26	15.56
ReAct2023.05		3.52	13.7	7.78	9.88	7.19	6.1	22.37	56	54.33	76.19	88.81	26.67	80	53.33	27.5	9.06	18.57	51.04	58.89	40.74	60	67.5	50	27.67	8	40.5	44	41	25.7	16.8	48.79	44.01	21.07	36.43
TDT2023.05		0.71	0.44	3.88	0.55	6.16	6.43	19.87	40.55	14.26	10.16	21.65	41.93	55.76	27.82	47.15	6.89	11.86	15.1	15.7	5.25	30	8.43	8.34	3.86	8	2.53	14.66	9.12	1.51	1.29	28.37	10.36	6.11	14.66
KGA2C2023.05		0	0	4	0	6	11	4	7	4	4	4	18	44	16	15	6	11	17	19	4	43	32	23	5	10	4	4	4	11	11	22.7	6.88	4.92	11.37
CALM2023.05		0	0	0	0	1	1	1	5	7	2	2	10	54	10	8	2	4	3	6	3	6	10	4	4	0	0	3	2	2	2	11.3	2.88	1.33	5.07