Symbolic Reasoning (Equations) on NoRa Irrelevant Rationales
49.3AccuracyCD-CoT
Evaluation Results
| Method | Links | |
|---|---|---|
| CD-CoTModel=Gemini-Pro2024.10 | 49.3 | |
| SCModel=Gemini-Pro2024.10 | 43.3 | |
| CD-CoTModel=GPT-3.5-turbo2024.10 | 42.7 | |
| BaseModel=Gemini-Pro2024.10 | 38.9 | |
| CCModel=Gemini-Pro2024.10 | 37.3 | |
| CCModel=GPT-3.5-turbo2024.10 | 32.7 | |
| BTModel=Gemini-Pro2024.10 | 29.3 | |
| SCModel=GPT-3.5-turbo2024.10 | 28.3 | |
| BaseModel=GPT-3.5-turbo2024.10 | 25.1 | |
| BTModel=GPT-3.5-turbo2024.10 | 22.7 | |
| CD-CoTModel=Mixtral 8x7B2024.10 | 22.7 | |
| SCModel=Mixtral 8x7B2024.10 | 21.7 | |
| CCModel=Mixtral 8x7B2024.10 | 18.5 | |
| BaseModel=Mixtral 8x7B2024.10 | 17.9 | |
| CCModel=LLaMA2-70B2024.10 | 16.3 | |
| BTModel=LLaMA2-70B2024.10 | 11.2 | |
| SCModel=LLaMA2-70B2024.10 | 10.3 | |
| CD-CoTModel=LLaMA2-70B2024.10 | 9.7 | |
| BTModel=Mixtral 8x7B2024.10 | 9.7 | |
| BaseModel=LLaMA2-70B2024.10 | 8.7 |