# Science Question Answering on ScienceQA v1.0 (test) Source: [Science Question Answering on ScienceQA v1.0 (test)]() Collection index: [llms.txt]() Data updated: 2026-02-26T05:34:16.488Z Evaluation of model performance on the ScienceQA v1.0 test set for Science Question Answering, reporting accuracy grouped by grade levels (G1-4, G5-8, G9-12) and overall average. A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. | Scope | Recorded value | | --- | --- | | Task | Science Question Answering | | Dataset (including any stated split) | ScienceQA v1.0 (test) | | Metric | Accuracy (G1-4) | | Unit | PERCENT | | Direction | Higher is better | | Results loaded | 26 of 26 | | Coverage | All results currently returned for this benchmark | Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol. ## Metrics - [Accuracy (G1-4)](): Higher is better; unit: PERCENT. - [Accuracy (G5-8)](): Higher is better; unit: PERCENT. - [Accuracy (G9-12)](): Higher is better; unit: PERCENT. - [Overall Accuracy](): Higher is better; unit: PERCENT. ## Recorded results | Method | Accuracy (G1-4) | Result date | Report paper | | --- | --- | --- | --- | | AUTOACT | 82.5 | Not recorded | 68fa73541e209131cea5bdd4 | | Chameleon | 77.83 | Not recorded | 68fa73541e209131cea5bdd4 | | Zero-Shot Plan\* | 76.67 | Not recorded | 68fa73541e209131cea5bdd4 | | Reflexion | 75.83 | Not recorded | 68fa73541e209131cea5bdd4 | | CoT | 74.17 | Not recorded | 68fa73541e209131cea5bdd4 | | FireAct | 72.5 | Not recorded | 68fa73541e209131cea5bdd4 | | AUTOACT | 70.83 | Not recorded | 68fa73541e209131cea5bdd4 | | BOLAA | 70 | Not recorded | 68fa73541e209131cea5bdd4 | | AUTOACT | 69.17 | Not recorded | 68fa73541e209131cea5bdd4 | | Chameleon | 69.17 | Not recorded | 68fa73541e209131cea5bdd4 | | Reflexion | 67.5 | Not recorded | 68fa73541e209131cea5bdd4 | | Reflexion | 67.5 | Not recorded | 68fa73541e209131cea5bdd4 | | Chameleon | 65.83 | Not recorded | 68fa73541e209131cea5bdd4 | | FireAct | 65 | Not recorded | 68fa73541e209131cea5bdd4 | | ReWOO | 65 | Not recorded | 68fa73541e209131cea5bdd4 | | BOLAA | 64.17 | Not recorded | 68fa73541e209131cea5bdd4 | | ReAct | 64.17 | Not recorded | 68fa73541e209131cea5bdd4 | | ReAct | 63.33 | Not recorded | 68fa73541e209131cea5bdd4 | | CoT | 61.67 | Not recorded | 68fa73541e209131cea5bdd4 | | CoT | 60.83 | Not recorded | 68fa73541e209131cea5bdd4 | | ReWOO | 60.83 | Not recorded | 68fa73541e209131cea5bdd4 | | FireAct | 60.83 | Not recorded | 68fa73541e209131cea5bdd4 | | BOLAA | 60 | Not recorded | 68fa73541e209131cea5bdd4 | | ReAct | 57.5 | Not recorded | 68fa73541e209131cea5bdd4 | | ReWOO | 57.5 | Not recorded | 68fa73541e209131cea5bdd4 | | CoT | 54.17 | Not recorded | 68fa73541e209131cea5bdd4 | ### AUTOACT Accuracy (G1-4): 82.5. Result ID: 6992bb853a6c3148491115e7. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Backbone":"Llama-2 70B-chat","Agent Learning Type":"Fine-tuning-based","Agent Architecture":"Multi-agent"}. This record does not assert independent reproduction. ### Chameleon Accuracy (G1-4): 77.83. Result ID: 6992bb853a6c3148491115e2. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Backbone":"Llama-2 70B-chat","Agent Learning Type":"Prompt-based","Agent Architecture":"Single-agent"}. This record does not assert independent reproduction. ### Zero-Shot Plan\* Accuracy (G1-4): 76.67. Result ID: 6992bb853a6c3148491115cf. [Reporting paper]() Recorded evaluation configuration: {"Backbone":"GPT-3.5 Turbo","Agent Learning Type":"Prompt-based","Agent Architecture":"Single-agent"}. This record does not assert independent reproduction. ### Reflexion Accuracy (G1-4): 75.83. Result ID: 6992bb853a6c3148491115e3. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Backbone":"Llama-2 70B-chat","Agent Learning Type":"Fine-tuning-based","Agent Architecture":"Single-agent"}. This record does not assert independent reproduction. ### CoT Accuracy (G1-4): 74.17. Result ID: 6992bb853a6c3148491115e0. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"Llama-2 70B-chat","Agent Learning Type":"Prompt-based","Agent Architecture":"Single-agent"}. This record does not assert independent reproduction. ### FireAct Accuracy (G1-4): 72.5. Result ID: 6992bb853a6c3148491115e6. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Backbone":"Llama-2 70B-chat","Agent Learning Type":"Fine-tuning-based","Agent Architecture":"Single-agent"}. This record does not assert independent reproduction. ### AUTOACT Accuracy (G1-4): 70.83. Result ID: 6992bb853a6c3148491115df. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Backbone":"Llama-2 13B-chat","Agent Learning Type":"Fine-tuning-based","Agent Architecture":"Multi-agent"}. This record does not assert independent reproduction. ### BOLAA Accuracy (G1-4): 70. Result ID: 6992bb853a6c3148491115e4. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"Llama-2 70B-chat","Agent Learning Type":"Prompt-based","Agent Architecture":"Multi-agent"}. This record does not assert independent reproduction. ### AUTOACT Accuracy (G1-4): 69.17. Result ID: 6992bb853a6c3148491115d7. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Backbone":"Mistral-7B Instruct v0.2","Agent Learning Type":"Fine-tuning-based","Agent Architecture":"Multi-agent"}. This record does not assert independent reproduction. ### Chameleon Accuracy (G1-4): 69.17. Result ID: 6992bb853a6c3148491115da. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Backbone":"Llama-2 13B-chat","Agent Learning Type":"Prompt-based","Agent Architecture":"Single-agent"}. This record does not assert independent reproduction. ### Reflexion Accuracy (G1-4): 67.5. Result ID: 6992bb853a6c3148491115d3. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Backbone":"Mistral-7B Instruct v0.2","Agent Learning Type":"Prompt-based","Agent Architecture":"Single-agent"}. This record does not assert independent reproduction. ### Reflexion Accuracy (G1-4): 67.5. Result ID: 6992bb853a6c3148491115db. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Backbone":"Llama-2 13B-chat","Agent Learning Type":"Fine-tuning-based","Agent Architecture":"Single-agent"}. This record does not assert independent reproduction. ### Chameleon Accuracy (G1-4): 65.83. Result ID: 6992bb853a6c3148491115d2. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Backbone":"Mistral-7B Instruct v0.2","Agent Learning Type":"Prompt-based","Agent Architecture":"Single-agent"}. This record does not assert independent reproduction. ### FireAct Accuracy (G1-4): 65. Result ID: 6992bb853a6c3148491115d6. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Backbone":"Mistral-7B Instruct v0.2","Agent Learning Type":"Fine-tuning-based","Agent Architecture":"Single-agent"}. This record does not assert independent reproduction. ### ReWOO Accuracy (G1-4): 65. Result ID: 6992bb853a6c3148491115e5. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"Llama-2 70B-chat","Agent Learning Type":"Prompt-based","Agent Architecture":"Single-agent"}. This record does not assert independent reproduction. ### BOLAA Accuracy (G1-4): 64.17. Result ID: 6992bb853a6c3148491115d4. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"Mistral-7B Instruct v0.2","Agent Learning Type":"Prompt-based","Agent Architecture":"Multi-agent"}. This record does not assert independent reproduction. ### ReAct Accuracy (G1-4): 64.17. Result ID: 6992bb853a6c3148491115e1. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Backbone":"Llama-2 70B-chat","Agent Learning Type":"Prompt-based","Agent Architecture":"Single-agent"}. This record does not assert independent reproduction. ### ReAct Accuracy (G1-4): 63.33. Result ID: 6992bb853a6c3148491115d1. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Backbone":"Mistral-7B Instruct v0.2","Agent Learning Type":"Prompt-based","Agent Architecture":"Single-agent"}. This record does not assert independent reproduction. ### CoT Accuracy (G1-4): 61.67. Result ID: 6992bb853a6c3148491115d8. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"Llama-2 13B-chat","Agent Learning Type":"Prompt-based","Agent Architecture":"Single-agent"}. This record does not assert independent reproduction. ### CoT Accuracy (G1-4): 60.83. Result ID: 6992bb853a6c3148491115ce. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"GPT-3.5 Turbo","Agent Learning Type":"Prompt-based","Agent Architecture":"Single-agent"}. This record does not assert independent reproduction. ### ReWOO Accuracy (G1-4): 60.83. Result ID: 6992bb853a6c3148491115d5. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"Mistral-7B Instruct v0.2","Agent Learning Type":"Prompt-based","Agent Architecture":"Single-agent"}. This record does not assert independent reproduction. ### FireAct Accuracy (G1-4): 60.83. Result ID: 6992bb853a6c3148491115de. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Backbone":"Llama-2 13B-chat","Agent Learning Type":"Fine-tuning-based","Agent Architecture":"Single-agent"}. This record does not assert independent reproduction. ### BOLAA Accuracy (G1-4): 60. Result ID: 6992bb853a6c3148491115dc. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"Llama-2 13B-chat","Agent Learning Type":"Prompt-based","Agent Architecture":"Multi-agent"}. This record does not assert independent reproduction. ### ReAct Accuracy (G1-4): 57.5. Result ID: 6992bb853a6c3148491115d9. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Backbone":"Llama-2 13B-chat","Agent Learning Type":"Prompt-based","Agent Architecture":"Single-agent"}. This record does not assert independent reproduction. ### ReWOO Accuracy (G1-4): 57.5. Result ID: 6992bb853a6c3148491115dd. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"Llama-2 13B-chat","Agent Learning Type":"Prompt-based","Agent Architecture":"Single-agent"}. This record does not assert independent reproduction. ### CoT Accuracy (G1-4): 54.17. Result ID: 6992bb853a6c3148491115d0. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"Mistral-7B Instruct v0.2","Agent Learning Type":"Prompt-based","Agent Architecture":"Single-agent"}. This record does not assert independent reproduction. Page 1 of 1. This is an index page, not the full collection.