Parts Mental Model Accuracy on ParRoT++
60.04Base LM AccuracyMacaw-11B
Evaluation Results
| Method | Links | |||
|---|---|---|---|---|
| Macaw-11B# params=11B2022.12 | 60.04 | 82.41 | 22.38 | |
| GPT-3 (text-davinci-003)# params=175B2022.12 | 55.51 | 71.13 | 15.62 |
Loading the SOTA2 catalog…
| Method | Links | |||
|---|---|---|---|---|
| Macaw-11B# params=11B2022.12 | 60.04 | 82.41 | 22.38 | |
| GPT-3 (text-davinci-003)# params=175B2022.12 | 55.51 | 71.13 | 15.62 |