# Knowledge Probing on Chemical Reactions

Source: [Knowledge Probing on Chemical Reactions](<https://www.sota2.com/research/sota/knowledge-probing-on-chemical-reactions>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2026-02-26T05:28:37.168Z

Evaluation of model performance on knowledge probing tasks using the Chemical Reactions dataset, measured by Accuracy.

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

| Scope | Recorded value |
| --- | --- |
| Task | Knowledge Probing |
| Dataset (including any stated split) | Chemical Reactions |
| Metric | Accuracy |
| Unit | PERCENT |
| Direction | Higher is better |
| Results loaded | 8 of 8 |
| Coverage | All results currently returned for this benchmark |

Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol.

## Metrics

- [Accuracy](<https://www.sota2.com/research/sota/knowledge-probing-on-chemical-reactions.md?metric=Accuracy>): Higher is better; unit: PERCENT.

## Recorded results

| Method | Accuracy | Result date | Report paper |
| --- | --- | --- | --- |
| GAL 120B | 43.1 | Not recorded | 68fa6dff1e209131cea3d479 |
| GAL 30B | 36.5 | Not recorded | 68fa6dff1e209131cea3d479 |
| GPT-3 (text-davinci-002) | 35.1 | Not recorded | 68fa6dff1e209131cea3d479 |
| GAL 6.7B | 26.4 | Not recorded | 68fa6dff1e209131cea3d479 |
| BLOOM | 22.4 | Not recorded | 68fa6dff1e209131cea3d479 |
| GAL 1.3B | 14.4 | Not recorded | 68fa6dff1e209131cea3d479 |
| OPT | 12.7 | Not recorded | 68fa6dff1e209131cea3d479 |
| GAL 125M | 0.3 | Not recorded | 68fa6dff1e209131cea3d479 |

### GAL 120B

Accuracy: 43.1. Result ID: 698ff26c3a6c314849fcfc77.

[Reporting paper](<https://www.sota2.com/research/paper/68fa6dff1e209131cea3d479.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa6dff1e209131cea3d479.md>)

[Implementation](<https://www.sota2.com/galactica.org>)

Recorded evaluation configuration: {"Params (bn)":120,"zero-shot":true}. This record does not assert independent reproduction.

### GAL 30B

Accuracy: 36.5. Result ID: 698ff26c3a6c314849fcfc76.

[Reporting paper](<https://www.sota2.com/research/paper/68fa6dff1e209131cea3d479.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa6dff1e209131cea3d479.md>)

[Implementation](<https://www.sota2.com/galactica.org>)

Recorded evaluation configuration: {"Params (bn)":30,"zero-shot":true}. This record does not assert independent reproduction.

### GPT-3 (text-davinci-002)

Accuracy: 35.1. Result ID: 698ff26c3a6c314849fcfc72.

[Reporting paper](<https://www.sota2.com/research/paper/68fa6dff1e209131cea3d479.md>)

Recorded evaluation configuration: {"Params (bn)":"?","zero-shot":true}. This record does not assert independent reproduction.

### GAL 6.7B

Accuracy: 26.4. Result ID: 698ff26c3a6c314849fcfc75.

[Reporting paper](<https://www.sota2.com/research/paper/68fa6dff1e209131cea3d479.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa6dff1e209131cea3d479.md>)

[Implementation](<https://www.sota2.com/galactica.org>)

Recorded evaluation configuration: {"Params (bn)":6.7,"zero-shot":true}. This record does not assert independent reproduction.

### BLOOM

Accuracy: 22.4. Result ID: 698ff26c3a6c314849fcfc71.

[Reporting paper](<https://www.sota2.com/research/paper/68fa6dff1e209131cea3d479.md>)

Recorded evaluation configuration: {"Params (bn)":176,"zero-shot":true}. This record does not assert independent reproduction.

### GAL 1.3B

Accuracy: 14.4. Result ID: 698ff26c3a6c314849fcfc74.

[Reporting paper](<https://www.sota2.com/research/paper/68fa6dff1e209131cea3d479.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa6dff1e209131cea3d479.md>)

[Implementation](<https://www.sota2.com/galactica.org>)

Recorded evaluation configuration: {"Params (bn)":1.3,"zero-shot":true}. This record does not assert independent reproduction.

### OPT

Accuracy: 12.7. Result ID: 698ff26c3a6c314849fcfc70.

[Reporting paper](<https://www.sota2.com/research/paper/68fa6dff1e209131cea3d479.md>)

Recorded evaluation configuration: {"Params (bn)":175,"zero-shot":true}. This record does not assert independent reproduction.

### GAL 125M

Accuracy: 0.3. Result ID: 698ff26c3a6c314849fcfc73.

[Reporting paper](<https://www.sota2.com/research/paper/68fa6dff1e209131cea3d479.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa6dff1e209131cea3d479.md>)

[Implementation](<https://www.sota2.com/galactica.org>)

Recorded evaluation configuration: {"Params (bn)":0.1,"zero-shot":true}. This record does not assert independent reproduction.

Page 1 of 1. This is an index page, not the full collection.