Mathematical Reasoning Skill Performance on MATH
56.4Overall PerformanceSKiC
Evaluation results
| Method | Links | |||||||||
|---|---|---|---|---|---|---|---|---|---|---|
| SKiCModel=GPT4, Ensemble=✘2023.08 | 56.4 | 79.7 | 43.6 | 29.5 | 74.6 | 58.2 | 36.6 | 55.9 | 24.3 | |
| ComplexCoTModel=GPT-4, Ensemble=PHP2023.08 | 53.9 | 73.8 | 41.9 | 26.3 | 73.4 | 56.3 | 29.8 | 55.7 | — | |
| VerificationModel=GPT-4, Ensemble=CSV2023.08 | 51.8 | 76.2 | 38.6 | 25.3 | 70.4 | 57 | 28.6 | 53.5 | — | |
| CoT, ScratchpadModel=Minerva-540B, Ensemble=maj1@k2023.08 | 50.3 | 71.1 | 42 | 27.1 | 72.7 | 43.5 | 34.5 | 36.3 | — | |
| ComplexCoTModel=GPT4, Ensemble=✘2023.08 | 50.3 | 71.6 | 36.5 | 23.4 | 70.8 | 53.1 | 26.7 | 49.6 | — | |
| CoTModel=PaLM-2, Ensemble=SC2023.08 | 48.8 | — | — | — | — | — | — | — | — | |
| CoTModel=GPT4, Ensemble=✘2023.08 | 42.2 | — | — | — | — | — | — | — | — | |
| SKiCModel=ChatGPT, Ensemble=✘2023.08 | 40.6 | 62 | 30.1 | 17.8 | 57.9 | 38.2 | 23 | 35.5 | 14.9 | |
| ComplexCoTModel=ChatGPT, Ensemble=PHP2023.08 | 36.5 | 57.7 | 25.4 | 17.1 | 49.1 | 33.7 | 16.1 | 35.1 | — | |
| VerificationModel=ChatGPT, Ensemble=CSV2023.08 | 34.7 | 58.9 | 22 | 14.8 | 45.6 | 35.2 | 13 | 33.5 | — | |
| CoTModel=PaLM-2, Ensemble=✘2023.08 | 34.3 | — | — | — | — | — | — | — | — | |
| ComplexCoTModel=ChatGPT, Ensemble=✘2023.08 | 34.1 | 53.8 | 22.3 | 14.6 | 49.1 | 29.7 | 16.8 | 33.4 | — | |
| CoT, ScratchpadModel=Minerva-540B, Ensemble=✘2023.08 | 33.6 | 54.9 | 26.7 | 13.6 | 51.2 | 27.9 | 18 | 21.2 | — | |
| CoT, LtPModel=ChatGPT, Ensemble=✘2023.08 | 31.1 | 52.3 | 22.5 | 16.9 | 49.6 | 30.2 | 16.3 | 29.8 | — |