Parts mental model prediction on ParRoT
0.7928AccuracyParRoT-Con (Macaw-11B)
Evaluation results
| Method | Links | ||
|---|---|---|---|
| ParRoT-Con (Macaw-11B)# params=11B, Constraint reasoning=Yes2022.12 | 0.7928 | 19.84 | |
| ParRoT-Con (GPT-3)# params=175B, Constraint reasoning=Yes2022.12 | 0.7026 | 16.43 | |
| Macaw-11B# params=11B, Constraint reasoning=No2022.12 | 0.5945 | — | |
| GPT-3 (text-davinci-003)# params=175B, Constraint reasoning=No2022.12 | 0.5383 | — |