Math Reasoning on ASDiv A (test)
91AccuracyATHENA
Evaluation results
| Method | Links | |
|---|---|---|
| ATHENABackbone=RoBERTa-large2023.11 | 91 | |
| CoReBackbone=GPT-J 12B, Evaluation Protocol=Zero-shot2022.10 | 90.5 | |
| DeductReasonerBackbone=RoBERTa-large2023.11 | 89.1 | |
| ATHENABackbone=RoBERTa-base2023.11 | 86.4 | |
| DeductReasonerBackbone=RoBERTa-base2023.11 | 83.1 | |
| R-Graph-to-TreeBackbone=RoBERTa-base2023.11 | 82.2 | |
| R-GTSBackbone=RoBERTa-base2023.11 | 81.2 | |
| Graph-to-TreeBackbone=Random embedding2023.11 | 77.4 | |
| Previous SoTAEvaluation Protocol=Fine-tune2022.10 | 75.3 | |
| R-TransformerBackbone=RoBERTa-base2023.11 | 72.1 | |
| GTSBackbone=Random embedding2023.11 | 71.4 | |
| Self-ConsistencyBackbone=GPT-J 6B, Evaluation Protocol=Zero-shot2022.10 | 63.7 | |
| TransformerBackbone=Random embedding2023.11 | 59.3 | |
| Generator onlyBackbone=GPT-J 6B, Evaluation Protocol=Zero-shot2022.10 | 51.7 |