Deductive Reasoning on ProofWriter (End-to-end Accuracy (%))
99.67End-to-end AccuracyD&P
Evaluation results
| Method | Links | |
|---|---|---|
| D&PParadigm=AF, Fallback Policy=Abstention (None), LLM=GPT-4o2026.03 | 99.67 | |
| D&PParadigm=AF, Fallback Policy=CoT, LLM=GPT-4o2026.03 | 99.67 | |
| LINCParadigm=AF, Fallback Policy=Abstention (None), LLM=GPT-4o2026.03 | 99.5 | |
| D&PParadigm=AF, Fallback Policy=Abstention (None), LLM=GPT-42026.03 | 98.33 | |
| D&PParadigm=AF, Fallback Policy=CoT, LLM=GPT-42026.03 | 98.33 | |
| LINCParadigm=AF, Fallback Policy=Abstention (None), LLM=GPT-42026.03 | 98.3 | |
| CLOVERParadigm=AF, Fallback Policy=CoT, LLM=GPT-4o2026.03 | 96.7 | |
| CLOVERParadigm=AF, Fallback Policy=Abstention (None), LLM=GPT-4o2026.03 | 96.5 | |
| MAD-LOGICParadigm=AF, Fallback Policy=Abstention (None), LLM=GPT-42026.03 | 92 | |
| SymbCoTParadigm=Prompting, Fallback Policy=N/A, LLM=GPT-42026.03 | 82.5 | |
| SymbCoTParadigm=Prompting, Fallback Policy=N/A, LLM=GPT-4o2026.03 | 80.2 | |
| Logic-LMParadigm=AF, Fallback Policy=CoT, LLM=GPT-42026.03 | 79.66 | |
| Logic-LM++Paradigm=AF, Fallback Policy=CoT, LLM=GPT-42026.03 | 79.66 | |
| DetermLRParadigm=Prompting, Fallback Policy=N/A, LLM=GPT-42026.03 | 79.17 | |
| Logic-LMParadigm=AF, Fallback Policy=Abstention (None), LLM=GPT-42026.03 | 79 | |
| Logic-LM++Paradigm=AF, Fallback Policy=Abstention (None), LLM=GPT-42026.03 | 78.8 | |
| CoTParadigm=Prompting, Fallback Policy=N/A, LLM=GPT-4o2026.03 | 72.67 | |
| CoT-SCParadigm=Prompting, Fallback Policy=N/A, LLM=GPT-42026.03 | 69.33 | |
| CoTParadigm=Prompting, Fallback Policy=N/A, LLM=GPT-42026.03 | 68.11 | |
| DirectParadigm=Prompting, Fallback Policy=N/A, LLM=GPT-4o2026.03 | 53.7 | |
| DirectParadigm=Prompting, Fallback Policy=N/A, LLM=GPT-42026.03 | 52.67 |