Multimodal Question Answering on ScienceQA v1.3 (test) benchmark leaderboard · SOTA2 Research

Multimodal Question Answering on ScienceQA v1.3 (test)

0.9019NAT ScoreFull Precision (Baseline)

Evaluation Results

Method	Links
Full Precision (Baseline)2024.10		0.9019	0.9314	0.8709	0.8939	0.8706	0.8983	0.9
Q-VLM2024.10		0.8954	0.9318	0.865	0.8812	0.8701	0.8885	0.897
Q-VLM2024.10		0.8943	0.9573	0.84	0.8871	0.8751	0.8725	0.8934
Full Precision (Baseline)2024.10		0.8939	0.9606	0.8564	0.8871	0.8765	0.885	0.8981
QLoRA2024.10		0.8887	0.9289	0.8564	0.8759	0.8656	0.8753	0.8887
QLoRA2024.10		0.8845	0.9471	0.8445	0.8763	0.8607	0.8787	0.8843
AWQ2024.10		0.8803	0.926	0.84	0.8602	0.8518	0.8641	0.8757
AWQ2024.10		0.8539	0.9201	0.8327	0.848	0.8354	0.8599	0.8623
Q-VLM2024.10		0.8255	0.7332	0.8318	0.8103	0.7082	0.8674	0.8078
Q-VLM2024.10		0.8086	0.7593	0.8073	0.8001	0.7248	0.839	0.7979
AWQ2024.10		0.8071	0.7061	0.7849	0.7946	0.7076	0.8182	0.7791
QLoRA2024.10		0.7962	0.7143	0.8245	0.7825	0.6842	0.853	0.7864
QLoRA2024.10		0.7753	0.7548	0.7918	0.7664	0.707	0.8195	0.7753
AWQ2024.10		0.7433	0.7222	0.7482	0.7341	0.6713	0.7798	0.7402
Q-VLM2024.10		0.6414	0.5868	0.6285	0.628	0.5523	0.6669	0.6246
Full Precision (Baseline)2024.10		0.6401	0.5857	0.633	0.628	0.5478	0.6697	0.6268
QLoRA2024.10		0.6298	0.5778	0.6185	0.6187	0.5399	0.6537	0.616
AWQ2024.10		0.5983	0.5778	0.6148	0.5894	0.5295	0.6425	0.5983
Q-VLM2024.10		0.5506	0.5194	0.5627	0.5457	0.4803	0.5834	0.5472
QLoRA2024.10		0.5448	0.4969	0.5555	0.532	0.473	0.5758	0.5324
AWQ2024.10		0.5369	0.4958	0.5427	0.5252	0.4781	0.5765	0.5298