Logical Reasoning on C-Judge
56.2AccuracyIAP-mv
Evaluation Results
| Method | Links | |
|---|---|---|
| IAP-mvBackbone=LLaMA-3 70B, Setting=Zero-shot CoT2024.09 | 56.2 | |
| Prompt #8Backbone=LLaMA-3 70B, Setting=Zero-shot CoT2024.09 | 55.61 | |
| Prompt #2Backbone=LLaMA-3 70B, Setting=Zero-shot CoT2024.09 | 54.55 | |
| Prompt #3Backbone=LLaMA-3 70B, Setting=Zero-shot CoT2024.09 | 49.73 | |
| Prompt #7Backbone=LLaMA-3 70B, Setting=Zero-shot CoT2024.09 | 47.59 | |
| Prompt #9Backbone=LLaMA-3 70B, Setting=Zero-shot CoT2024.09 | 44.92 | |
| Prompt #6Backbone=LLaMA-3 70B, Setting=Zero-shot CoT2024.09 | 44.39 | |
| Prompt #4Backbone=LLaMA-3 70B, Setting=Zero-shot CoT2024.09 | 42.25 | |
| Prompt #1Backbone=LLaMA-3 70B, Setting=Zero-shot CoT2024.09 | 38.5 | |
| Prompt #5Backbone=LLaMA-3 70B, Setting=Zero-shot CoT2024.09 | 36.36 | |
| IAP-mvBackbone=LLaMA-2-13B-Chat2024.09 | 33.47 | |
| Prompt #6Backbone=LLaMA-2-13B-Chat, Strategy=Zero-shot CoT2024.09 | 32.68 | |
| IAP-ssBackbone=LLaMA-2-13B-Chat2024.09 | 32.56 | |
| Prompt #3Backbone=LLaMA-2-13B-Chat, Strategy=Zero-shot CoT2024.09 | 24.06 | |
| Prompt #4Backbone=LLaMA-2-13B-Chat, Strategy=Zero-shot CoT2024.09 | 16.58 | |
| Prompt #5Backbone=LLaMA-2-13B-Chat, Strategy=Zero-shot CoT2024.09 | 14.97 | |
| Prompt #7Backbone=LLaMA-2-13B-Chat, Strategy=Zero-shot CoT2024.09 | 14.97 | |
| Prompt #2Backbone=LLaMA-2-13B-Chat, Strategy=Zero-shot CoT2024.09 | 13.9 | |
| OPPRBackbone=LLaMA-2-13B-Chat2024.09 | 13.37 | |
| Prompt #1Backbone=LLaMA-2-13B-Chat, Strategy=Zero-shot CoT2024.09 | 11.76 | |
| Self-discBackbone=LLaMA-2-13B-Chat2024.09 | 10.16 | |
| Prompt #9Backbone=LLaMA-2-13B-Chat, Strategy=Zero-shot CoT2024.09 | 9.62 | |
| Prompt #8Backbone=LLaMA-2-13B-Chat, Strategy=Zero-shot CoT2024.09 | 7.49 |