Common Sense Reasoning on NoRa Irrelevant Rationales
60AccuracySC
Evaluation results
| Method | Links | |
|---|---|---|
| SCModel=Gemini-Pro2024.10 | 60 | |
| CD-CoTModel=Gemini-Pro2024.10 | 57.7 | |
| CD-CoTModel=Gemini-Pro2024.10 | 55.7 | |
| CD-CoTModel=GPT-3.5-turbo2024.10 | 54.7 | |
| BaseModel=Gemini-Pro2024.10 | 53.2 | |
| CCModel=Gemini-Pro2024.10 | 50.2 | |
| CD-CoTModel=GPT-3.5-turbo2024.10 | 49.7 | |
| SCModel=LLaMA2-70B2024.10 | 46.7 | |
| SCModel=LLaMA2-70B2024.10 | 46 | |
| CCModel=Gemini-Pro2024.10 | 45.6 | |
| SCModel=GPT-3.5-turbo2024.10 | 45 | |
| SCModel=GPT-3.5-turbo2024.10 | 44.7 | |
| CCModel=GPT-3.5-turbo2024.10 | 43.6 | |
| SCModel=Gemini-Pro2024.10 | 42.7 | |
| BaseModel=GPT-3.5-turbo2024.10 | 42.3 | |
| BaseModel=LLaMA2-70B2024.10 | 41.9 | |
| CCModel=GPT-3.5-turbo2024.10 | 40.8 | |
| CD-CoTModel=Mixtral 8x7B2024.10 | 40.3 | |
| CD-CoTModel=Mixtral 8x7B2024.10 | 40.3 | |
| BaseModel=LLaMA2-70B2024.10 | 40.2 | |
| CD-CoTModel=LLaMA2-70B2024.10 | 39.7 | |
| CD-CoTModel=LLaMA2-70B2024.10 | 39.3 | |
| BTModel=Gemini-Pro2024.10 | 37.8 | |
| SCModel=Mixtral 8x7B2024.10 | 37.7 | |
| SCModel=Mixtral 8x7B2024.10 | 37 | |
| BTModel=LLaMA2-70B2024.10 | 36.2 | |
| BTModel=LLaMA2-70B2024.10 | 36.1 | |
| CCModel=Mixtral 8x7B2024.10 | 36 | |
| CCModel=Mixtral 8x7B2024.10 | 35.7 | |
| BaseModel=Gemini-Pro2024.10 | 35.5 | |
| BaseModel=Mixtral 8x7B2024.10 | 34.9 | |
| BaseModel=GPT-3.5-turbo2024.10 | 33.4 | |
| BTModel=Gemini-Pro2024.10 | 33.3 | |
| BaseModel=Mixtral 8x7B2024.10 | 31.1 | |
| CCModel=LLaMA2-70B2024.10 | 29.9 | |
| BTModel=GPT-3.5-turbo2024.10 | 28.3 | |
| CCModel=LLaMA2-70B2024.10 | 28.3 | |
| BTModel=GPT-3.5-turbo2024.10 | 11 | |
| BTModel=Mixtral 8x7B2024.10 | 10.5 | |
| BTModel=Mixtral 8x7B2024.10 | 6.2 |