Mathematical Reasoning on Math Base-9
92.7AccuracyCD-CoT
Evaluation results
| Method | Links | ||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|
| CD-CoTModel=Gemini-Pro2024.10 | 92.7 | — | — | — | — | — | — | — | — | — | |
| BTModel=Gemini-Pro2024.10 | 82.4 | — | — | — | — | — | — | — | — | — | |
| SCModel=Gemini-Pro2024.10 | 80.3 | — | — | — | — | — | — | — | — | — | |
| BaseModel=Gemini-Pro2024.10 | 72.3 | — | — | — | — | — | — | — | — | — | |
| CCModel=Gemini-Pro2024.10 | 67.5 | — | — | — | — | — | — | — | — | — | |
| CD-CoTModel=GPT-3.5-turbo2024.10 | 60.7 | — | — | — | — | — | — | — | — | — | |
| CCModel=GPT-3.5-turbo2024.10 | 44.3 | — | — | — | — | — | — | — | — | — | |
| SCModel=GPT-3.5-turbo2024.10 | 36.6 | — | — | — | — | — | — | — | — | — | |
| BTModel=GPT-3.5-turbo2024.10 | 34.2 | — | — | — | — | — | — | — | — | — | |
| BaseModel=GPT-3.5-turbo2024.10 | 30.3 | — | — | — | — | — | — | — | — | — | |
| CCModel=Mixtral 8x7B2024.10 | 24.4 | — | — | — | — | — | — | — | — | — | |
| SCModel=Mixtral 8x7B2024.10 | 20 | — | — | — | — | — | — | — | — | — | |
| BaseModel=Mixtral 8x7B2024.10 | 16.3 | — | — | — | — | — | — | — | — | — | |
| CD-CoTModel=Mixtral 8x7B2024.10 | 8.7 | — | — | — | — | — | — | — | — | — | |
| SCModel=LLaMA2-70B2024.10 | 5 | — | — | — | — | — | — | — | — | — | |
| BTModel=Mixtral 8x7B2024.10 | 4.1 | — | — | — | — | — | — | — | — | — | |
| CD-CoTModel=LLaMA2-70B2024.10 | 4 | — | — | — | — | — | — | — | — | — | |
| BaseModel=LLaMA2-70B2024.10 | 2.8 | — | — | — | — | — | — | — | — | — | |
| BTModel=LLaMA2-70B2024.10 | 1.4 | — | — | — | — | — | — | — | — | — | |
| CCModel=LLaMA2-70B2024.10 | 1.1 | — | — | — | — | — | — | — | — | — | |
| Base2024.10 | — | 46.4 | 39.3 | 30.3 | 26.6 | 32.1 | 23.2 | 10.1 | 6 | 13.1 | |
| BTAdditional Information=Noise Position2024.10 | — | 47.2 | 39.2 | 34.2 | 29.9 | 34.4 | 30.1 | 18.4 | 14.1 | 20.9 | |
| CCAdditional Information=Clean Demo2024.10 | — | 44.9 | 43.3 | 44.6 | 45.5 | 44.5 | 37.2 | 31.7 | 30.7 | 33.2 | |
| CD-CoTAdditional Information=Clean Demo2024.10 | — | 60.7 | 59.7 | 60.7 | 57.2 | 59.2 | 54 | 58.7 | 48.4 | 53.7 | |
| SCOAdditional Information=Ground Truth2024.10 | — | 53.6 | 46.3 | 39.6 | 36.4 | 40.8 | 34.7 | 22 | 17.7 | 24.8 |