Full proof generation on NATURALPROOFS-GEN core (test)
23.52Reasoning Errors (Ref.)NATURALPROVER_RETRIEVE
Evaluation results
| Method | Links | |||||||||
|---|---|---|---|---|---|---|---|---|---|---|
| NATURALPROVER_RETRIEVEKnowledge source=Retrieved (top-20), Decoding strategy=Greedy, Backbone=GPT-3 (Curie)2022.05 | 23.52 | 37.55 | 23.66 | 4.54 | 6.19 | 41.54 | 33.56 | 32 | 24 | |
| NATURALPROVER++Knowledge source=Human-provided, Decoding strategy=Stepwise++, Backbone=GPT-3 (Curie)2022.05 | 23.61 | 28.54 | 18.45 | 5.58 | 3.65 | 46.57 | 35.41 | 45 | 32 | |
| NATURALPROVERKnowledge source=Human-provided, Decoding strategy=Greedy, Backbone=GPT-3 (Curie)2022.05 | 25.84 | 35.93 | 25.23 | 8.41 | 5.35 | 39.6 | 26.3 | 35 | 24 | |
| GPT-3Knowledge source=None, Decoding strategy=Greedy, Backbone=GPT-3 (Curie)2022.05 | 30.92 | 32.54 | 40.15 | 5.61 | 5.24 | 25.69 | 28.18 | 20 | 13 |