Multimodal Science Question Answering on ScienceQA v1.0 (test) benchmark leaderboard · SOTA2 Research

Multimodal Science Question Answering on ScienceQA v1.0 (test)

97.47Accuracy (Natural Language Component)MC-CoT_F-Large

Evaluation Results

Method	Links
MC-CoT_F-Large2023.11		97.47	90.44	93.18	96.97	93.75	94.49	95.3	94.13	94.88
Mutimodal-CoT_Large2023.11		95.91	82	90.82	95.26	88.8	92.89	92.44	90.31	91.68
MC-CoT_Large2023.11		95.47	89.99	91.82	95.11	92.66	93.24	94.27	91.76	93.37
MC-CoT_F-Base2023.11		93.56	83.58	90.73	94.13	89.24	90.94	90.93	90.38	90.73
MC-CoT_Base2023.11		91.87	84.59	93	92.28	88.3	92.75	90.64	90.64	90.64
LLaVa + GPT-42023.11		91.56	96.74	91.09	90.62	88.99	93.52	92.73	92.16	92.53
LLaVa2023.11		90.36	95.95	88	89.49	88	90.66	90.93	90.9	90.92
LaVIN-13B2023.11		90.32	94.38	87.73	89.44	87.65	90.31	91.19	89.26	90.5
Human2023.11		90.23	84.97	87.48	89.6	87.5	88.1	91.59	82.42	88.4
Chameleon + GPT-42023.11		89.83	74.13	89.82	88.27	77.64	92.13	88.03	83.72	86.54
LLaMA-SciTune_Large2023.11		89.3	95.61	87	93.08	86.67	91.75	84.37	91.3	90.03
LaVIN-7B2023.11		89.25	94.94	85.24	88.51	87.46	88.08	90.16	88.07	89.41
Mutimodal-CoT_Base2023.11		87.52	77.17	85.82	87.88	82.9	86.83	84.65	85.37	84.91
GPT-4 w/ CoT2023.11		85.48	72.44	90.27	82.65	71.49	92.89	86.66	79.04	83.99
LLaMA-SciTune_Base2023.11		84.5	94.15	82.91	88.35	83.64	88.74	85.05	85.6	86.11
LLaMA-Adapter2023.11		84.37	88.3	84.36	83.72	80.32	86.9	85.83	84.05	85.19
Chameleon + ChatGPT2023.11		81.62	70.64	84	79.77	70.8	86.62	81.86	76.53	79.93
LLaMA-Adapter (T)2023.11		79	73.79	80.55	78.3	70.35	83.14	79.77	75.68	78.31
ChatGPT w/ CoT2023.11		78.82	70.98	83.18	77.37	67.92	86.13	80.72	74.03	78.31
GPT-3.5 w/ CoT2023.11		75.44	70.87	78.09	74.68	67.43	79.93	78.23	69.68	75.17
GPT-3.52023.11		74.64	69.74	76	74.44	67.28	77.42	76.8	68.89	73.97
UnifiedQA_Base w/ COT2023.11		71	76.04	78.91	66.42	66.53	81.81	77.06	68.82	74.11
UnifiedQA_Base2023.11		68.16	69.18	74.91	63.78	61.38	77.84	72.98	65	70.12
Patch-TRM2023.11		65.19	46.79	65.55	66.96	55.28	64.95	58.04	67.5	61.42
DFAF2023.11		64.03	48.82	63.55	65.88	54.49	64.11	57.12	67.17	60.72
BAN2023.11		60.88	46.57	66.64	62.61	52.6	65.51	56.83	63.94	59.37
VILT2023.11		60.48	63.89	60.27	63.2	61.38	57	60.72	61.9	61.14
Top-Down2023.11		59.5	54.33	61.82	62.9	54.88	59.79	57.27	62.16	59.02
VisualBERT2023.11		59.33	69.18	61.18	62.71	62.17	58.54	62.96	59.92	61.87
MCAN2023.11		56.08	46.23	58.09	59.43	51.17	55.4	51.65	59.72	54.54
Random Choice2023.11		40.28	46.13	29.25	47.75	40.08	33.66	39.35	40.67	39.83