# Science Question Answering on ScienceQA (test)

Source: [Science Question Answering on ScienceQA (test)](<https://www.sota2.com/research/sota/science-question-answering-on-scienceqa-test>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2026-06-01T07:43:54.434Z

Evaluation of model performance on the test split of the ScienceQA dataset for Science Question Answering, utilizing detailed metrics covering Natural Language, Symbolic Output, and Image/Text components.

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

| Scope | Recorded value |
| --- | --- |
| Task | Science Question Answering |
| Dataset (including any stated split) | ScienceQA (test) |
| Metric | Average Accuracy |
| Unit | PERCENT |
| Direction | Higher is better |
| Results loaded | 311 of 311 |
| Coverage | All results currently returned for this benchmark |

Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol.

## Metrics

- [Average Accuracy](<https://www.sota2.com/research/sota/science-question-answering-on-scienceqa-test.md?metric=Average%20Accuracy>): Higher is better; unit: PERCENT.

- [NAT Accuracy](<https://www.sota2.com/research/sota/science-question-answering-on-scienceqa-test.md?metric=NAT%20Accuracy>): Higher is better; unit: PERCENT.

- [SOC Accuracy](<https://www.sota2.com/research/sota/science-question-answering-on-scienceqa-test.md?metric=SOC%20Accuracy>): Higher is better; unit: PERCENT.

- [LAN Accuracy](<https://www.sota2.com/research/sota/science-question-answering-on-scienceqa-test.md?metric=LAN%20Accuracy>): Higher is better; unit: PERCENT.

- [TXT Accuracy](<https://www.sota2.com/research/sota/science-question-answering-on-scienceqa-test.md?metric=TXT%20Accuracy>): Higher is better; unit: PERCENT.

- [IMG Accuracy](<https://www.sota2.com/research/sota/science-question-answering-on-scienceqa-test.md?metric=IMG%20Accuracy>): Higher is better; unit: PERCENT.

- [NO Accuracy](<https://www.sota2.com/research/sota/science-question-answering-on-scienceqa-test.md?metric=NO%20Accuracy>): Higher is better; unit: PERCENT.

- [Grade 1-6 Accuracy](<https://www.sota2.com/research/sota/science-question-answering-on-scienceqa-test.md?metric=Grade%201-6%20Accuracy>): Higher is better; unit: PERCENT.

- [Grade 7-12 Accuracy](<https://www.sota2.com/research/sota/science-question-answering-on-scienceqa-test.md?metric=Grade%207-12%20Accuracy>): Higher is better; unit: PERCENT.

- [Overall Score](<https://www.sota2.com/research/sota/science-question-answering-on-scienceqa-test.md?metric=Overall%20Score>): Higher is better; unit: PERCENT.

- [Performance Retention](<https://www.sota2.com/research/sota/science-question-answering-on-scienceqa-test.md?metric=Performance%20Retention>): Higher is better; unit: PERCENT.

- [Sum](<https://www.sota2.com/research/sota/science-question-answering-on-scienceqa-test.md?metric=Sum>): Higher is better; unit: SCALAR.

- [R Overall Score](<https://www.sota2.com/research/sota/science-question-answering-on-scienceqa-test.md?metric=R%20Overall%20Score>): Higher is better; unit: SCALAR.

- [R Accuracy Score](<https://www.sota2.com/research/sota/science-question-answering-on-scienceqa-test.md?metric=R%20Accuracy%20Score>): Higher is better; unit: SCALAR.

- [R Reasoning Score](<https://www.sota2.com/research/sota/science-question-answering-on-scienceqa-test.md?metric=R%20Reasoning%20Score>): Higher is better; unit: SCALAR.

- [R Comprehensive Score](<https://www.sota2.com/research/sota/science-question-answering-on-scienceqa-test.md?metric=R%20Comprehensive%20Score>): Higher is better; unit: SCALAR.

- [R Pedagogic Score](<https://www.sota2.com/research/sota/science-question-answering-on-scienceqa-test.md?metric=R%20Pedagogic%20Score>): Higher is better; unit: SCALAR.

- [R Confidence Score](<https://www.sota2.com/research/sota/science-question-answering-on-scienceqa-test.md?metric=R%20Confidence%20Score>): Higher is better; unit: SCALAR.

- [Performance Drop (%)](<https://www.sota2.com/research/sota/science-question-answering-on-scienceqa-test.md?metric=Performance%20Drop%20(%25)>): Lower is better; unit: PERCENT.

## Recorded results

| Method | Average Accuracy | Result date | Report paper |
| --- | --- | --- | --- |
| Perceptio | 98.3 | Not recorded | 69bcd4a10636e5bf4afc2b0b |
| PVC InternVL2 (Ours) | 97.7 | Not recorded | 68fa76261e209131cea6b803 |
| Perceptio | 97.2 | Not recorded | 69bcd4a10636e5bf4afc2b0b |
| InternVL2 | 97.1 | Not recorded | 68fa76261e209131cea6b803 |
| Sa2VA | 96.8 | Not recorded | 69bcd4a10636e5bf4afc2b0b |
| ReProbe | 96.7 | Not recorded | 6912c2ee2828b7cc6937d9ec |
| ReProbe | 96.2 | Not recorded | 6912c2ee2828b7cc6937d9ec |
| LLaVA-OV | 96 | Not recorded | 68fa76261e209131cea6b803 |
| Sa2VA | 95.8 | Not recorded | 69bcd4a10636e5bf4afc2b0b |
| Swift | 95.02 | Not recorded | 68fa79421e209131cea7cecd |
| PVC InternVL2 (Ours) | 94.9 | Not recorded | 68fa76261e209131cea6b803 |
| Refit | 94.53 | Not recorded | 68fa79421e209131cea7cecd |
| Honeybee (M=576) | 94.39 | Not recorded | 68fa72c01e209131cea585db |
| InternVL2-2B | 94.1 | Not recorded | 68fa75051e209131cea65504 |
| InternVL2 | 94 | Not recorded | 68fa76261e209131cea6b803 |
| MR-MKG (FLAN-UL2-19B) | 93.63 | Not recorded | 68fa73711e209131cea5c853 |
| Honeybee (M=256) | 93.19 | Not recorded | 68fa72c01e209131cea585db |
| ReProbe | 92.8 | Not recorded | 6912c2ee2828b7cc6937d9ec |
| MR-MKG (FLAN-T5-11B) | 92.78 | Not recorded | 68fa73711e209131cea5c853 |
| CogVLM | 92.7 | Not recorded | 68fa72ba1e209131cea5830e |
| LLaVA+GPT-4 | 92.53 | Not recorded | 68fa717d1e209131cea51970 |
| LLaVA+GPT-4 (judge) | 92.53 | Not recorded | 68fa72c01e209131cea585db |
| CaMML-13B | 92.03 | Not recorded | 68fa73bd1e209131cea5e2bd |
| MM1.5-30B | 91.9 | Not recorded | 68fa75051e209131cea65504 |
| MM-COT Large | 91.68 | Not recorded | 68fa70f81e209131cea4e731 |
| MM-CoT\_Large | 91.68 | Not recorded | 68fa710e1e209131cea4eee8 |
| MM-COT | 91.68 | Not recorded | 68fa717d1e209131cea51970 |
| MM-COT | 91.68 | Not recorded | 68fa72c01e209131cea585db |
| UnifiedQA Large (MM-CoT) | 91.68 | Not recorded | 68fa73711e209131cea5c853 |
| MMCoT Large | 91.68 | Not recorded | 68fa73bd1e209131cea5e2bd |
| CaMML-7B | 91.32 | Not recorded | 68fa73bd1e209131cea5e2bd |
| Qwen2.5-Math-7B-PRM800k | 91.2 | Not recorded | 6912c2ee2828b7cc6937d9ec |
| LLaVA+GPT-4 | 90.97 | Not recorded | 68fa717d1e209131cea51970 |
| LLaVA | 90.92 | Not recorded | 68fa70f81e209131cea4e731 |
| LLaVA | 90.92 | Not recorded | 68fa717d1e209131cea51970 |
| LLaVA | 90.92 | Not recorded | 68fa72c01e209131cea585db |
| LLaVA | 90.92 | Not recorded | 68fa72d31e209131cea58bff |
| LLaVA | 90.92 | Not recorded | 68fa73421e209131cea5b5b0 |
| LLaVA | 90.92 | Not recorded | 68fa73711e209131cea5c853 |
| LLaVA-13B | 90.9 | Not recorded | 68fa73bd1e209131cea5e2bd |
| LaVIN-13B† | 90.83 | Not recorded | 68fa70f81e209131cea4e731 |
| Phi-3-Vision-4B | 90.8 | Not recorded | 68fa75051e209131cea65504 |
| ReProbe | 90.8 | Not recorded | 6912c2ee2828b7cc6937d9ec |
| InstructBLIP | 90.7 | Not recorded | 68fa72ba1e209131cea5830e |
| GPT-4o | 90.7 | Not recorded | 68fa75051e209131cea65504 |
| LaVIN-13B | 90.5 | Not recorded | 68fa70f81e209131cea4e731 |
| Multimodal-CoT Large | 90.45 | Not recorded | 68fa73421e209131cea5b5b0 |
| STaR-GATE | 90.05 | Not recorded | 68fa79421e209131cea7cecd |
| STaR-GATE-D | 90.05 | Not recorded | 68fa79421e209131cea7cecd |
| LaVIN-13B | 90.03 | Not recorded | 68fa73711e209131cea5c853 |
| LLaVA-SQA-7B | 89.91 | Not recorded | 68fa77861e209131cea73d16 |
| MM1.5-3B-MoE | 89.8 | Not recorded | 68fa75051e209131cea65504 |
| MM1.5-7B | 89.6 | Not recorded | 68fa75051e209131cea65504 |
| Qwen2.5-VL-7B + Cont. Reward | 89.5 | Not recorded | 691ff210d6b0dd334292e6db |
| EfficientLLaVA | 89.46 | Not recorded | 68fa77861e209131cea73d16 |
| LaVIN-7B | 89.41 | Not recorded | 68fa70f81e209131cea4e731 |
| LaVIN-7B | 89.41 | Not recorded | 68fa73711e209131cea5c853 |
| LLaVA-7B | 89.28 | Not recorded | 68fa73bd1e209131cea5e2bd |
| SparseGPT | 88.99 | Not recorded | 68fa77861e209131cea73d16 |
| Chat-UniVi | 88.78 | Not recorded | 68fa72d31e209131cea58bff |
| EfficientLLaVA | 88.73 | Not recorded | 68fa77861e209131cea73d16 |
| Vision-Zero | 88.5 | Not recorded | 691ff210d6b0dd334292e6db |
| MR-MKG (FLAN-T5-3B) | 88.47 | Not recorded | 68fa73711e209131cea5c853 |
| Human | 88.4 | Not recorded | 68fa70f81e209131cea4e731 |
| Human | 88.4 | Not recorded | 68fa710e1e209131cea4eee8 |
| Human | 88.4 | Not recorded | 68fa717d1e209131cea51970 |
| Human | 88.4 | Not recorded | 68fa72c01e209131cea585db |
| Human | 88.4 | Not recorded | 68fa72d31e209131cea58bff |
| Human | 88.4 | Not recorded | 68fa73421e209131cea5b5b0 |
| Human | 88.4 | Not recorded | 68fa73711e209131cea5c853 |
| Human Average | 88.4 | Not recorded | 68fa73bd1e209131cea5e2bd |
| BLIP-3 | 88.3 | Not recorded | 68fa75051e209131cea65504 |
| Qwen2.5-VL-7B (Baseline) | 88.3 | Not recorded | 691ff210d6b0dd334292e6db |
| SmoGVLM | 88.02 | Not recorded | 69e7048f7d0172140ec7ecdd |
| Qwen2.5-VL-7B + Discrete Reward | 87.98 | Not recorded | 691ff210d6b0dd334292e6db |
| MM1.5-1B-MoE | 87.6 | Not recorded | 68fa75051e209131cea65504 |
| LLaVA | 87.1 | Not recorded | 69e7048f7d0172140ec7ecdd |
| SmoGVLM | 86.87 | Not recorded | 69e7048f7d0172140ec7ecdd |
| Chameleon (GPT-4) | 86.54 | Not recorded | 68fa710e1e209131cea4eee8 |
| Chameleon (GPT-4) | 86.54 | Not recorded | 68fa73421e209131cea5b5b0 |

### Perceptio

Average Accuracy: 98.3. Result ID: 69bd1bed47f34ce924a7c80b.

[Reporting paper](<https://www.sota2.com/research/paper/69bcd4a10636e5bf4afc2b0b.md>)

[Method origin paper](<https://www.sota2.com/research/paper/69bcd4a10636e5bf4afc2b0b.md>)

Recorded evaluation configuration: {"Parameters":"8B"}. This record does not assert independent reproduction.

### PVC InternVL2 (Ours)

Average Accuracy: 97.7. Result ID: 699411d83a6c3148491ad40b.

[Reporting paper](<https://www.sota2.com/research/paper/68fa76261e209131cea6b803.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa76261e209131cea6b803.md>)

[Implementation](<https://github.com/OpenGVLab/PVC>)

Recorded evaluation configuration: {"Size":"8B","#token /image tile":"64"}. This record does not assert independent reproduction.

### Perceptio

Average Accuracy: 97.2. Result ID: 69bd1bed47f34ce924a7c80a.

[Reporting paper](<https://www.sota2.com/research/paper/69bcd4a10636e5bf4afc2b0b.md>)

[Method origin paper](<https://www.sota2.com/research/paper/69bcd4a10636e5bf4afc2b0b.md>)

Recorded evaluation configuration: {"Parameters":"4B"}. This record does not assert independent reproduction.

### InternVL2

Average Accuracy: 97.1. Result ID: 699411d83a6c3148491ad40a.

[Reporting paper](<https://www.sota2.com/research/paper/68fa76261e209131cea6b803.md>)

Recorded evaluation configuration: {"Size":"8B","#token /image tile":"256"}. This record does not assert independent reproduction.

### Sa2VA

Average Accuracy: 96.8. Result ID: 69bd1bed47f34ce924a7c809.

[Reporting paper](<https://www.sota2.com/research/paper/69bcd4a10636e5bf4afc2b0b.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa76ec1e209131cea6ff2c.md>)

Recorded evaluation configuration: {"Parameters":"8B"}. This record does not assert independent reproduction.

### ReProbe

Average Accuracy: 96.7. Result ID: 69e9edd747f34ce924d19c9f.

[Reporting paper](<https://www.sota2.com/research/paper/6912c2ee2828b7cc6937d9ec.md>)

[Method origin paper](<https://www.sota2.com/research/paper/6912c2ee2828b7cc6937d9ec.md>)

Recorded evaluation configuration: {"# Sample":"32K","Input Features":"Hidden States","Annotation Source":"DeepSeek-anno","Decoding Method":"Beam search"}. This record does not assert independent reproduction.

### ReProbe

Average Accuracy: 96.2. Result ID: 69e9edd747f34ce924d19c9c.

[Reporting paper](<https://www.sota2.com/research/paper/6912c2ee2828b7cc6937d9ec.md>)

[Method origin paper](<https://www.sota2.com/research/paper/6912c2ee2828b7cc6937d9ec.md>)

Recorded evaluation configuration: {"# Sample":"32K","Input Features":"Attn+Logit","Annotation Source":"Self-anno","Decoding Method":"Beam search"}. This record does not assert independent reproduction.

### LLaVA-OV

Average Accuracy: 96. Result ID: 699411d83a6c3148491ad409.

[Reporting paper](<https://www.sota2.com/research/paper/68fa76261e209131cea6b803.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa756f1e209131cea67e58.md>)

[Implementation](<https://llava-vl.github.io/blog/llava-onevision>)

Recorded evaluation configuration: {"Size":"7B","#token /image tile":"729"}. This record does not assert independent reproduction.

### Sa2VA

Average Accuracy: 95.8. Result ID: 69bd1bed47f34ce924a7c808.

[Reporting paper](<https://www.sota2.com/research/paper/69bcd4a10636e5bf4afc2b0b.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa76ec1e209131cea6ff2c.md>)

Recorded evaluation configuration: {"Parameters":"4B"}. This record does not assert independent reproduction.

### Swift

Average Accuracy: 95.02. Result ID: 699a86f35698665ec410f8f3.

[Reporting paper](<https://www.sota2.com/research/paper/68fa79421e209131cea7cecd.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa79421e209131cea7cecd.md>)

Recorded evaluation configuration: {"Mode":"Standard Mode","Thinking (%)":"0.0 ± 0.0"}. This record does not assert independent reproduction.

### PVC InternVL2 (Ours)

Average Accuracy: 94.9. Result ID: 699411d83a6c3148491ad408.

[Reporting paper](<https://www.sota2.com/research/paper/68fa76261e209131cea6b803.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa76261e209131cea6b803.md>)

[Implementation](<https://github.com/OpenGVLab/PVC>)

Recorded evaluation configuration: {"Size":"2B","#token /image tile":"64"}. This record does not assert independent reproduction.

### Refit

Average Accuracy: 94.53. Result ID: 699a86f35698665ec410f8f4.

[Reporting paper](<https://www.sota2.com/research/paper/68fa79421e209131cea7cecd.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa79421e209131cea7cecd.md>)

Recorded evaluation configuration: {"Mode":"Standard Mode","Thinking (%)":"0.0 ± 0.0"}. This record does not assert independent reproduction.

### Honeybee (M=576)

Average Accuracy: 94.39. Result ID: 699277963a6c3148490f28d3.

[Reporting paper](<https://www.sota2.com/research/paper/68fa72c01e209131cea585db.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa72c01e209131cea585db.md>)

[Implementation](<https://github.com/kakaobrain/honeybee>)

Recorded evaluation configuration: {"Model Category":"Generalist Models","Number of visual tokens (M)":576}. This record does not assert independent reproduction.

### InternVL2-2B

Average Accuracy: 94.1. Result ID: 699370673a6c31484915f0c6.

[Reporting paper](<https://www.sota2.com/research/paper/68fa75051e209131cea65504.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa73081e209131cea59f8b.md>)

[Implementation](<https://github.com/OpenGVLab/InternVL>)

Recorded evaluation configuration: {"Model Scale":"1B","Zero-shot":true,"Decoding Strategy":"beam search"}. This record does not assert independent reproduction.

### InternVL2

Average Accuracy: 94. Result ID: 699411d83a6c3148491ad407.

[Reporting paper](<https://www.sota2.com/research/paper/68fa76261e209131cea6b803.md>)

Recorded evaluation configuration: {"Size":"2B","#token /image tile":"256"}. This record does not assert independent reproduction.

### MR-MKG (FLAN-UL2-19B)

Average Accuracy: 93.63. Result ID: 6992df5e3a6c3148491211a9.

[Reporting paper](<https://www.sota2.com/research/paper/68fa73711e209131cea5c853.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa73711e209131cea5c853.md>)

Recorded evaluation configuration: {"#T-Param":"248M","Backbone":"FLAN-UL2-19B"}. This record does not assert independent reproduction.

### Honeybee (M=256)

Average Accuracy: 93.19. Result ID: 699277963a6c3148490f28d2.

[Reporting paper](<https://www.sota2.com/research/paper/68fa72c01e209131cea585db.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa72c01e209131cea585db.md>)

[Implementation](<https://github.com/kakaobrain/honeybee>)

Recorded evaluation configuration: {"Model Category":"Generalist Models","Number of visual tokens (M)":256}. This record does not assert independent reproduction.

### ReProbe

Average Accuracy: 92.8. Result ID: 69e9edd747f34ce924d19c9e.

[Reporting paper](<https://www.sota2.com/research/paper/6912c2ee2828b7cc6937d9ec.md>)

[Method origin paper](<https://www.sota2.com/research/paper/6912c2ee2828b7cc6937d9ec.md>)

Recorded evaluation configuration: {"# Sample":"32K","Input Features":"Hidden States","Annotation Source":"Self-anno","Decoding Method":"Beam search"}. This record does not assert independent reproduction.

### MR-MKG (FLAN-T5-11B)

Average Accuracy: 92.78. Result ID: 6992df5e3a6c3148491211a8.

[Reporting paper](<https://www.sota2.com/research/paper/68fa73711e209131cea5c853.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa73711e209131cea5c853.md>)

Recorded evaluation configuration: {"#T-Param":"248M","Backbone":"FLAN-T5-11B"}. This record does not assert independent reproduction.

### CogVLM

Average Accuracy: 92.7. Result ID: 699272f43a6c3148490f052a.

[Reporting paper](<https://www.sota2.com/research/paper/68fa72ba1e209131cea5830e.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa721c1e209131cea548b9.md>)

[Implementation](<https://github.com/THUDM/CogVLM>)

Recorded evaluation configuration: {"Category":"Specialist"}. This record does not assert independent reproduction.

### LLaVA+GPT-4

Average Accuracy: 92.53. Result ID: 6991e4cf3a6c3148490b4778.

[Reporting paper](<https://www.sota2.com/research/paper/68fa717d1e209131cea51970.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa717d1e209131cea51970.md>)

[Implementation](<https://llava-vl.github.io/>)

Recorded evaluation configuration: {"Ensemble strategy":"judge"}. This record does not assert independent reproduction.

### LLaVA+GPT-4 (judge)

Average Accuracy: 92.53. Result ID: 699277963a6c3148490f28d1.

[Reporting paper](<https://www.sota2.com/research/paper/68fa72c01e209131cea585db.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa717d1e209131cea51970.md>)

[Implementation](<https://llava-vl.github.io/>)

Recorded evaluation configuration: {"Model Category":"Specialist Models","Evaluation Judge":"GPT-4"}. This record does not assert independent reproduction.

### CaMML-13B

Average Accuracy: 92.03. Result ID: 69931aa93a6c31484913bc56.

[Reporting paper](<https://www.sota2.com/research/paper/68fa73bd1e209131cea5e2bd.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa73bd1e209131cea5e2bd.md>)

[Implementation](<https://github.com/amazon-science/camml>)

Recorded evaluation configuration: {"Backbone":"Vicuna-13B"}. This record does not assert independent reproduction.

### MM1.5-30B

Average Accuracy: 91.9. Result ID: 699370673a6c31484915f0d9.

[Reporting paper](<https://www.sota2.com/research/paper/68fa75051e209131cea65504.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa75051e209131cea65504.md>)

[Implementation](<https://github.com/apple/axlearn>)

Recorded evaluation configuration: {"Model Scale":"30B","Zero-shot":true,"Decoding Strategy":"greedy"}. This record does not assert independent reproduction.

### MM-COT Large

Average Accuracy: 91.68. Result ID: 699169273a6c31484907f5e5.

[Reporting paper](<https://www.sota2.com/research/paper/68fa70f81e209131cea4e731.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa73421e209131cea5b5b0.md>)

[Implementation](<https://github.com/amazon-science/mm-cot>)

Recorded evaluation configuration: {"#T-Params":"738M","LLM":false}. This record does not assert independent reproduction.

### MM-CoT\_Large

Average Accuracy: 91.68. Result ID: 699178e63a6c314849085eea.

[Reporting paper](<https://www.sota2.com/research/paper/68fa710e1e209131cea4eee8.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa73421e209131cea5b5b0.md>)

[Implementation](<https://github.com/amazon-science/mm-cot>)

Recorded evaluation configuration: {"#Tuned Params":"738M"}. This record does not assert independent reproduction.

### MM-COT

Average Accuracy: 91.68. Result ID: 6991e4cf3a6c3148490b4774.

[Reporting paper](<https://www.sota2.com/research/paper/68fa717d1e209131cea51970.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa73421e209131cea5b5b0.md>)

[Implementation](<https://github.com/amazon-science/mm-cot>)

Recorded evaluation configuration: {"Model size":"Large"}. This record does not assert independent reproduction.

### MM-COT

Average Accuracy: 91.68. Result ID: 699277963a6c3148490f28cf.

[Reporting paper](<https://www.sota2.com/research/paper/68fa72c01e209131cea585db.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa73421e209131cea5b5b0.md>)

[Implementation](<https://github.com/amazon-science/mm-cot>)

Recorded evaluation configuration: {"Model Category":"Specialist Models"}. This record does not assert independent reproduction.

### UnifiedQA Large (MM-CoT)

Average Accuracy: 91.68. Result ID: 6992df5e3a6c3148491211a2.

[Reporting paper](<https://www.sota2.com/research/paper/68fa73711e209131cea5c853.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa73421e209131cea5b5b0.md>)

[Implementation](<https://github.com/amazon-science/mm-cot>)

Recorded evaluation configuration: {"#T-Param":"738M","Prompting":"MM-CoT"}. This record does not assert independent reproduction.

### MMCoT Large

Average Accuracy: 91.68. Result ID: 69931aa93a6c31484913bc52.

[Reporting paper](<https://www.sota2.com/research/paper/68fa73bd1e209131cea5e2bd.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa73421e209131cea5b5b0.md>)

[Implementation](<https://github.com/amazon-science/mm-cot>)

Recorded evaluation configuration: {"Size":"Large"}. This record does not assert independent reproduction.

### CaMML-7B

Average Accuracy: 91.32. Result ID: 69931aa93a6c31484913bc55.

[Reporting paper](<https://www.sota2.com/research/paper/68fa73bd1e209131cea5e2bd.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa73bd1e209131cea5e2bd.md>)

[Implementation](<https://github.com/amazon-science/camml>)

Recorded evaluation configuration: {"Backbone":"Vicuna-7B"}. This record does not assert independent reproduction.

### Qwen2.5-Math-7B-PRM800k

Average Accuracy: 91.2. Result ID: 69e9edd747f34ce924d19c9a.

[Reporting paper](<https://www.sota2.com/research/paper/6912c2ee2828b7cc6937d9ec.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa6f9a1e209131cea468c7.md>)

[Implementation](<https://github.com/openai/prm800k>)

Recorded evaluation configuration: {"# Sample":"263K","Decoding Method":"Beam search"}. This record does not assert independent reproduction.

### LLaVA+GPT-4

Average Accuracy: 90.97. Result ID: 6991e4cf3a6c3148490b4777.

[Reporting paper](<https://www.sota2.com/research/paper/68fa717d1e209131cea51970.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa717d1e209131cea51970.md>)

[Implementation](<https://llava-vl.github.io/>)

Recorded evaluation configuration: {"Ensemble strategy":"complement"}. This record does not assert independent reproduction.

### LLaVA

Average Accuracy: 90.92. Result ID: 699169273a6c31484907f5e6.

[Reporting paper](<https://www.sota2.com/research/paper/68fa70f81e209131cea4e731.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa717d1e209131cea51970.md>)

[Implementation](<https://llava-vl.github.io/>)

Recorded evaluation configuration: {"#T-Params":"13B","LLM":true}. This record does not assert independent reproduction.

### LLaVA

Average Accuracy: 90.92. Result ID: 6991e4cf3a6c3148490b4776.

[Reporting paper](<https://www.sota2.com/research/paper/68fa717d1e209131cea51970.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa717d1e209131cea51970.md>)

[Implementation](<https://llava-vl.github.io/>)

Recorded evaluation configuration: {"Model size":"13B"}. This record does not assert independent reproduction.

### LLaVA

Average Accuracy: 90.92. Result ID: 699277963a6c3148490f28d0.

[Reporting paper](<https://www.sota2.com/research/paper/68fa72c01e209131cea585db.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa717d1e209131cea51970.md>)

[Implementation](<https://llava-vl.github.io/>)

Recorded evaluation configuration: {"Model Category":"Specialist Models"}. This record does not assert independent reproduction.

### LLaVA

Average Accuracy: 90.92. Result ID: 69929aba3a6c314849101854.

[Reporting paper](<https://www.sota2.com/research/paper/68fa72d31e209131cea58bff.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa717d1e209131cea51970.md>)

[Implementation](<https://llava-vl.github.io/>)

Recorded evaluation configuration: {"LLM Size":"13B","Evaluation Protocol":"Fine-tuning"}. This record does not assert independent reproduction.

### LLaVA

Average Accuracy: 90.92. Result ID: 6992b3b23a6c31484910dc0e.

[Reporting paper](<https://www.sota2.com/research/paper/68fa73421e209131cea5b5b0.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa717d1e209131cea51970.md>)

[Implementation](<https://llava-vl.github.io/>)

Recorded evaluation configuration: {"Size":"13B"}. This record does not assert independent reproduction.

### LLaVA

Average Accuracy: 90.92. Result ID: 6992df5e3a6c3148491211a3.

[Reporting paper](<https://www.sota2.com/research/paper/68fa73711e209131cea5c853.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa717d1e209131cea51970.md>)

[Implementation](<https://llava-vl.github.io/>)

Recorded evaluation configuration: {"#T-Param":"13B"}. This record does not assert independent reproduction.

### LLaVA-13B

Average Accuracy: 90.9. Result ID: 69931aa93a6c31484913bc54.

[Reporting paper](<https://www.sota2.com/research/paper/68fa73bd1e209131cea5e2bd.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa717d1e209131cea51970.md>)

[Implementation](<https://llava-vl.github.io/>)

Recorded evaluation configuration: {"Backbone":"Vicuna-13B"}. This record does not assert independent reproduction.

### LaVIN-13B†

Average Accuracy: 90.83. Result ID: 699169273a6c31484907f5ea.

[Reporting paper](<https://www.sota2.com/research/paper/68fa70f81e209131cea4e731.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa70f81e209131cea4e731.md>)

[Implementation](<https://luogen1996.github.io/lavin>)

Recorded evaluation configuration: {"#T-Params":"5.4M","LLM":true,"trained\_epochs":40}. This record does not assert independent reproduction.

### Phi-3-Vision-4B

Average Accuracy: 90.8. Result ID: 699370673a6c31484915f0cf.

[Reporting paper](<https://www.sota2.com/research/paper/68fa75051e209131cea65504.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa74ae1e209131cea632e1.md>)

Recorded evaluation configuration: {"Model Scale":"3B","Zero-shot":true,"Decoding Strategy":"greedy"}. This record does not assert independent reproduction.

### ReProbe

Average Accuracy: 90.8. Result ID: 69e9edd747f34ce924d19c9d.

[Reporting paper](<https://www.sota2.com/research/paper/6912c2ee2828b7cc6937d9ec.md>)

[Method origin paper](<https://www.sota2.com/research/paper/6912c2ee2828b7cc6937d9ec.md>)

Recorded evaluation configuration: {"# Sample":"32K","Input Features":"Attn+Logit","Annotation Source":"DeepSeek-anno","Decoding Method":"Beam search"}. This record does not assert independent reproduction.

### InstructBLIP

Average Accuracy: 90.7. Result ID: 699272f43a6c3148490f0529.

[Reporting paper](<https://www.sota2.com/research/paper/68fa72ba1e209131cea5830e.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa6fb71e209131cea47021.md>)

[Implementation](<https://github.com/salesforce/LAVIS/tree/main/projects/instructblip>)

Recorded evaluation configuration: {"Category":"Specialist"}. This record does not assert independent reproduction.

### GPT-4o

Average Accuracy: 90.7. Result ID: 699370673a6c31484915f0dc.

[Reporting paper](<https://www.sota2.com/research/paper/68fa75051e209131cea65504.md>)

Recorded evaluation configuration: {"Zero-shot":true,"Decoding Strategy":"greedy"}. This record does not assert independent reproduction.

### LaVIN-13B

Average Accuracy: 90.5. Result ID: 699169273a6c31484907f5e9.

[Reporting paper](<https://www.sota2.com/research/paper/68fa70f81e209131cea4e731.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa70f81e209131cea4e731.md>)

[Implementation](<https://luogen1996.github.io/lavin>)

Recorded evaluation configuration: {"#T-Params":"5.4M","LLM":true}. This record does not assert independent reproduction.

### Multimodal-CoT Large

Average Accuracy: 90.45. Result ID: 6992b3b23a6c31484910dc11.

[Reporting paper](<https://www.sota2.com/research/paper/68fa73421e209131cea5b5b0.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa73421e209131cea5b5b0.md>)

[Implementation](<https://github.com/amazon-science/mm-cot>)

Recorded evaluation configuration: {"Size":"738M"}. This record does not assert independent reproduction.

### STaR-GATE

Average Accuracy: 90.05. Result ID: 699a86f35698665ec410f8f6.

[Reporting paper](<https://www.sota2.com/research/paper/68fa79421e209131cea7cecd.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa74621e209131cea6159f.md>)

Recorded evaluation configuration: {"Mode":"Standard Mode","Thinking (%)":"0.0 ± 0.0"}. This record does not assert independent reproduction.

### STaR-GATE-D

Average Accuracy: 90.05. Result ID: 699a86f35698665ec410f8f8.

[Reporting paper](<https://www.sota2.com/research/paper/68fa79421e209131cea7cecd.md>)

Recorded evaluation configuration: {"Mode":"Standard Mode","Thinking (%)":"0.0 ± 0.0"}. This record does not assert independent reproduction.

### LaVIN-13B

Average Accuracy: 90.03. Result ID: 6992df5e3a6c3148491211a6.

[Reporting paper](<https://www.sota2.com/research/paper/68fa73711e209131cea5c853.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa70f81e209131cea4e731.md>)

[Implementation](<https://luogen1996.github.io/lavin>)

Recorded evaluation configuration: {"#T-Param":"5.4M","Backbone":"LLaMA-13B"}. This record does not assert independent reproduction.

### LLaVA-SQA-7B

Average Accuracy: 89.91. Result ID: 69952c8c3badfaffe612cfd9.

[Reporting paper](<https://www.sota2.com/research/paper/68fa77861e209131cea73d16.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa717d1e209131cea51970.md>)

[Implementation](<https://llava-vl.github.io/>)

Recorded evaluation configuration: {"Architecture":"LLaVA-SQA-7B","Ratio":0}. This record does not assert independent reproduction.

### MM1.5-3B-MoE

Average Accuracy: 89.8. Result ID: 699370673a6c31484915f0d2.

[Reporting paper](<https://www.sota2.com/research/paper/68fa75051e209131cea65504.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa75051e209131cea65504.md>)

[Implementation](<https://github.com/apple/axlearn>)

Recorded evaluation configuration: {"Model Scale":"3B","Architecture":"MoE","Zero-shot":true,"Decoding Strategy":"greedy"}. This record does not assert independent reproduction.

### MM1.5-7B

Average Accuracy: 89.6. Result ID: 699370673a6c31484915f0d5.

[Reporting paper](<https://www.sota2.com/research/paper/68fa75051e209131cea65504.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa75051e209131cea65504.md>)

[Implementation](<https://github.com/apple/axlearn>)

Recorded evaluation configuration: {"Model Scale":"7B","Zero-shot":true,"Decoding Strategy":"greedy"}. This record does not assert independent reproduction.

### Qwen2.5-VL-7B + Cont. Reward

Average Accuracy: 89.5. Result ID: 69b7ddd647f34ce924a04a24.

[Reporting paper](<https://www.sota2.com/research/paper/691ff210d6b0dd334292e6db.md>)

[Method origin paper](<https://www.sota2.com/research/paper/691ff210d6b0dd334292e6db.md>)

Recorded evaluation configuration: {"reward\_type":"Continuous"}. This record does not assert independent reproduction.

### EfficientLLaVA

Average Accuracy: 89.46. Result ID: 69952c8c3badfaffe612cfdc.

[Reporting paper](<https://www.sota2.com/research/paper/68fa77861e209131cea73d16.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa77861e209131cea73d16.md>)

Recorded evaluation configuration: {"Architecture":"LLaVA-SQA-7B","Ratio":0.3}. This record does not assert independent reproduction.

### LaVIN-7B

Average Accuracy: 89.41. Result ID: 699169273a6c31484907f5e8.

[Reporting paper](<https://www.sota2.com/research/paper/68fa70f81e209131cea4e731.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa70f81e209131cea4e731.md>)

[Implementation](<https://luogen1996.github.io/lavin>)

Recorded evaluation configuration: {"#T-Params":"3.8M","LLM":true}. This record does not assert independent reproduction.

### LaVIN-7B

Average Accuracy: 89.41. Result ID: 6992df5e3a6c3148491211a5.

[Reporting paper](<https://www.sota2.com/research/paper/68fa73711e209131cea5c853.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa70f81e209131cea4e731.md>)

[Implementation](<https://luogen1996.github.io/lavin>)

Recorded evaluation configuration: {"#T-Param":"3.8M","Backbone":"LLaMA-7B"}. This record does not assert independent reproduction.

### LLaVA-7B

Average Accuracy: 89.28. Result ID: 69931aa93a6c31484913bc53.

[Reporting paper](<https://www.sota2.com/research/paper/68fa73bd1e209131cea5e2bd.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa717d1e209131cea51970.md>)

[Implementation](<https://llava-vl.github.io/>)

Recorded evaluation configuration: {"Backbone":"Vicuna-7B"}. This record does not assert independent reproduction.

### SparseGPT

Average Accuracy: 88.99. Result ID: 69952c8c3badfaffe612cfdb.

[Reporting paper](<https://www.sota2.com/research/paper/68fa77861e209131cea73d16.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa6ecc1e209131cea42371.md>)

[Implementation](<https://github.com/IST-DASLab/sparsegpt>)

Recorded evaluation configuration: {"Architecture":"LLaVA-SQA-7B","Ratio":0.3}. This record does not assert independent reproduction.

### Chat-UniVi

Average Accuracy: 88.78. Result ID: 69929aba3a6c314849101858.

[Reporting paper](<https://www.sota2.com/research/paper/68fa72d31e209131cea58bff.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa72d31e209131cea58bff.md>)

[Implementation](<https://github.com/PKU-YuanGroup/Chat-UniVi>)

Recorded evaluation configuration: {"LLM Size":"7B","Evaluation Protocol":"Fine-tuning"}. This record does not assert independent reproduction.

### EfficientLLaVA

Average Accuracy: 88.73. Result ID: 69952c8c3badfaffe612cfdf.

[Reporting paper](<https://www.sota2.com/research/paper/68fa77861e209131cea73d16.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa77861e209131cea73d16.md>)

Recorded evaluation configuration: {"Architecture":"LLaVA-SQA-7B","Ratio":0.4}. This record does not assert independent reproduction.

### Vision-Zero

Average Accuracy: 88.5. Result ID: 69b7ddd647f34ce924a04a21.

[Reporting paper](<https://www.sota2.com/research/paper/691ff210d6b0dd334292e6db.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa7bb11e209131cea89eff.md>)

Recorded evaluation configuration: {"external supervision":"true"}. This record does not assert independent reproduction.

### MR-MKG (FLAN-T5-3B)

Average Accuracy: 88.47. Result ID: 6992df5e3a6c3148491211a7.

[Reporting paper](<https://www.sota2.com/research/paper/68fa73711e209131cea5c853.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa73711e209131cea5c853.md>)

Recorded evaluation configuration: {"#T-Param":"77M","Backbone":"FLAN-T5-3B"}. This record does not assert independent reproduction.

### Human

Average Accuracy: 88.4. Result ID: 699169273a6c31484907f5df.

[Reporting paper](<https://www.sota2.com/research/paper/68fa70f81e209131cea4e731.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa6dac1e209131cea3b607.md>)

[Implementation](<https://scienceqa.github.io/>)

Recorded evaluation configuration: {"LLM":false}. This record does not assert independent reproduction.

### Human

Average Accuracy: 88.4. Result ID: 699178e63a6c314849085ede.

[Reporting paper](<https://www.sota2.com/research/paper/68fa710e1e209131cea4eee8.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa6dac1e209131cea3b607.md>)

[Implementation](<https://scienceqa.github.io/>)

Recorded evaluation configuration: Not recorded. This record does not assert independent reproduction.

### Human

Average Accuracy: 88.4. Result ID: 6991e4cf3a6c3148490b476f.

[Reporting paper](<https://www.sota2.com/research/paper/68fa717d1e209131cea51970.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa6dac1e209131cea3b607.md>)

[Implementation](<https://scienceqa.github.io/>)

Recorded evaluation configuration: Not recorded. This record does not assert independent reproduction.

### Human

Average Accuracy: 88.4. Result ID: 699277963a6c3148490f28cb.

[Reporting paper](<https://www.sota2.com/research/paper/68fa72c01e209131cea585db.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa6dac1e209131cea3b607.md>)

[Implementation](<https://scienceqa.github.io/>)

Recorded evaluation configuration: {"type":"Human baseline"}. This record does not assert independent reproduction.

### Human

Average Accuracy: 88.4. Result ID: 69929aba3a6c31484910184f.

[Reporting paper](<https://www.sota2.com/research/paper/68fa72d31e209131cea58bff.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa6dac1e209131cea3b607.md>)

[Implementation](<https://scienceqa.github.io/>)

Recorded evaluation configuration: {"Evaluation Protocol":"Baseline"}. This record does not assert independent reproduction.

### Human

Average Accuracy: 88.4. Result ID: 6992b3b23a6c31484910dbfe.

[Reporting paper](<https://www.sota2.com/research/paper/68fa73421e209131cea5b5b0.md>)

Recorded evaluation configuration: Not recorded. This record does not assert independent reproduction.

### Human

Average Accuracy: 88.4. Result ID: 6992df5e3a6c31484912119d.

[Reporting paper](<https://www.sota2.com/research/paper/68fa73711e209131cea5c853.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa6dac1e209131cea3b607.md>)

[Implementation](<https://scienceqa.github.io/>)

Recorded evaluation configuration: Not recorded. This record does not assert independent reproduction.

### Human Average

Average Accuracy: 88.4. Result ID: 69931aa93a6c31484913bc4c.

[Reporting paper](<https://www.sota2.com/research/paper/68fa73bd1e209131cea5e2bd.md>)

Recorded evaluation configuration: Not recorded. This record does not assert independent reproduction.

### BLIP-3

Average Accuracy: 88.3. Result ID: 699370673a6c31484915f0ce.

[Reporting paper](<https://www.sota2.com/research/paper/68fa75051e209131cea65504.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa7b671e209131cea8836c.md>)

Recorded evaluation configuration: {"Model Scale":"3B","Zero-shot":true,"Decoding Strategy":"greedy"}. This record does not assert independent reproduction.

### Qwen2.5-VL-7B (Baseline)

Average Accuracy: 88.3. Result ID: 69b7ddd647f34ce924a04a22.

[Reporting paper](<https://www.sota2.com/research/paper/691ff210d6b0dd334292e6db.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa773a1e209131cea71d3c.md>)

Recorded evaluation configuration: Not recorded. This record does not assert independent reproduction.

### SmoGVLM

Average Accuracy: 88.02. Result ID: 69e7287447f34ce924cd0588.

[Reporting paper](<https://www.sota2.com/research/paper/69e7048f7d0172140ec7ecdd.md>)

[Method origin paper](<https://www.sota2.com/research/paper/69e7048f7d0172140ec7ecdd.md>)

Recorded evaluation configuration: {"Size":"13B","Method":"FFT"}. This record does not assert independent reproduction.

### Qwen2.5-VL-7B + Discrete Reward

Average Accuracy: 87.98. Result ID: 69b7ddd647f34ce924a04a23.

[Reporting paper](<https://www.sota2.com/research/paper/691ff210d6b0dd334292e6db.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa7b2d1e209131cea8707e.md>)

Recorded evaluation configuration: {"reward\_type":"Discrete"}. This record does not assert independent reproduction.

### MM1.5-1B-MoE

Average Accuracy: 87.6. Result ID: 699370673a6c31484915f0c9.

[Reporting paper](<https://www.sota2.com/research/paper/68fa75051e209131cea65504.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa75051e209131cea65504.md>)

[Implementation](<https://github.com/apple/axlearn>)

Recorded evaluation configuration: {"Model Scale":"1B","Architecture":"MoE","Zero-shot":true,"Decoding Strategy":"greedy"}. This record does not assert independent reproduction.

### LLaVA

Average Accuracy: 87.1. Result ID: 69e7287447f34ce924cd0586.

[Reporting paper](<https://www.sota2.com/research/paper/69e7048f7d0172140ec7ecdd.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa717d1e209131cea51970.md>)

[Implementation](<https://llava-vl.github.io/>)

Recorded evaluation configuration: {"Size":"13B","Method":"FFT"}. This record does not assert independent reproduction.

### SmoGVLM

Average Accuracy: 86.87. Result ID: 69e7287447f34ce924cd0584.

[Reporting paper](<https://www.sota2.com/research/paper/69e7048f7d0172140ec7ecdd.md>)

[Method origin paper](<https://www.sota2.com/research/paper/69e7048f7d0172140ec7ecdd.md>)

Recorded evaluation configuration: {"Size":"7B","Method":"FFT"}. This record does not assert independent reproduction.

### Chameleon (GPT-4)

Average Accuracy: 86.54. Result ID: 699178e63a6c314849085ef2.

[Reporting paper](<https://www.sota2.com/research/paper/68fa710e1e209131cea4eee8.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa710e1e209131cea4eee8.md>)

[Implementation](<https://chameleon-llm.github.io/>)

Recorded evaluation configuration: {"#Tuned Params":"0M","Few-shot":true,"Base LLM":"GPT-4"}. This record does not assert independent reproduction.

### Chameleon (GPT-4)

Average Accuracy: 86.54. Result ID: 6992b3b23a6c31484910dc0c.

[Reporting paper](<https://www.sota2.com/research/paper/68fa73421e209131cea5b5b0.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa710e1e209131cea4eee8.md>)

[Implementation](<https://chameleon-llm.github.io/>)

Recorded evaluation configuration: Not recorded. This record does not assert independent reproduction.

Page 1 of 4. This is an index page, not the full collection.

- [Next index page](<https://www.sota2.com/research/sota/science-question-answering-on-scienceqa-test.md?metric=Average%20Accuracy&page=2>): Follow this link to discover more pages.
