Symbolic Reasoning (Equations) on NoRa Inaccurate Rationales
53.3AccuracyCD-CoT
Evaluation Results
| Method | Links | |
|---|---|---|
| CD-CoTModel=Gemini-Pro2024.10 | 53.3 | |
| SCModel=Gemini-Pro2024.10 | 45 | |
| CD-CoTModel=GPT-3.5-turbo2024.10 | 41.3 | |
| BaseModel=Gemini-Pro2024.10 | 36.7 | |
| CCModel=Gemini-Pro2024.10 | 35 | |
| CCModel=GPT-3.5-turbo2024.10 | 33 | |
| SCModel=GPT-3.5-turbo2024.10 | 30.7 | |
| BTModel=Gemini-Pro2024.10 | 28.7 | |
| BaseModel=GPT-3.5-turbo2024.10 | 26.1 | |
| BTModel=GPT-3.5-turbo2024.10 | 22.7 | |
| CD-CoTModel=Mixtral 8x7B2024.10 | 21.3 | |
| CCModel=Mixtral 8x7B2024.10 | 18.3 | |
| SCModel=Mixtral 8x7B2024.10 | 18 | |
| BaseModel=Mixtral 8x7B2024.10 | 15.1 | |
| CCModel=LLaMA2-70B2024.10 | 14 | |
| BTModel=LLaMA2-70B2024.10 | 12.5 | |
| BTModel=Mixtral 8x7B2024.10 | 10.1 | |
| SCModel=LLaMA2-70B2024.10 | 9.7 | |
| CD-CoTModel=LLaMA2-70B2024.10 | 9.7 | |
| BaseModel=LLaMA2-70B2024.10 | 9.1 |