LaTeX Equation Generation on LaTeX Equations Knowledge Probe 434 equations (test)
79.6Chemistry AccuracyGAL 120B
Evaluation results
| Method | Links | ||||||
|---|---|---|---|---|---|---|---|
| GAL 120BParams (bn)=120, Evaluation=zero-shot2022.11 | 79.6 | 83.5 | 72.4 | 52.5 | 36.4 | 68.2 | |
| GAL 30BParams (bn)=30, Evaluation=zero-shot2022.11 | 63.6 | 74.4 | 35.2 | 40.4 | 34.1 | 51.5 | |
| GPT-3 (text-davinci-002)Evaluation=zero-shot2022.11 | 61.4 | 65.4 | 41.9 | 25.3 | 31.8 | 49 | |
| GAL 6.7BParams (bn)=6.7, Evaluation=zero-shot2022.11 | 43.2 | 59.4 | 36.2 | 29.3 | 27.3 | 41.7 | |
| BLOOMParams (bn)=176, Evaluation=zero-shot2022.11 | 36.3 | 36.1 | 6.6 | 14.1 | 13.6 | 21.4 | |
| OPTParams (bn)=175, Evaluation=zero-shot2022.11 | 34.1 | 4.5 | 22.9 | 1 | 2.3 | 8.9 | |
| GAL 1.3BParams (bn)=1.3, Evaluation=zero-shot2022.11 | 31.8 | 26.3 | 23.8 | 11.1 | 4.6 | 20.5 | |
| GAL 125MParams (bn)=0.1, Evaluation=zero-shot2022.11 | 0 | 0.8 | 0 | 1 | 0 | 0.5 |