Logical Reasoning on ProofWriter CWA 3-hop
73.8AccuracyLSS-CoT
Evaluation results
| Method | Links | ||
|---|---|---|---|
| LSS-CoTModel=Gemma-2-9B It2026.04 | 73.8 | 2.4 | |
| 3-shot-CoTModel=Gemma-2-9B It2026.04 | 72.8 | — | |
| SC-3Model=Gemma-2-9B It2026.04 | 72.8 | — | |
| Greedy-CoTModel=Gemma-2-9B It2026.04 | 71.4 | — | |
| 3-shot-CoTModel=Phi-3-mini-4k Instruct2026.04 | 69.2 | — | |
| 3-shot-CoTModel=Llama-3.1-8B Instruct2026.04 | 66.4 | — | |
| SC-3Model=Phi-3-mini-4k Instruct2026.04 | 66.4 | — | |
| LSS-CoTModel=Phi-3-mini-4k Instruct2026.04 | 63.4 | 3.8 | |
| Greedy-CoTModel=Phi-3-mini-4k Instruct2026.04 | 59.6 | — | |
| 3-shot-CoTModel=LLaMA-2-13B Chat2026.04 | 58 | — | |
| 3-shot-CoTModel=LLaMA-3.2-3B Instruct2026.04 | 56.8 | — | |
| LSS-CoTModel=LLaMA-2-13B Chat2026.04 | 56.6 | 2 | |
| LSS-CoTModel=Llama-3.1-8B Instruct2026.04 | 55.6 | 4.2 | |
| Greedy-CoTModel=LLaMA-2-13B Chat2026.04 | 54.6 | — | |
| SC-3Model=LLaMA-2-13B Chat2026.04 | 52.6 | — | |
| Greedy-CoTModel=Llama-3.1-8B Instruct2026.04 | 51.4 | — | |
| LSS-CoTModel=LLaMA-3.2-3B Instruct2026.04 | 49.8 | 3.2 | |
| SC-3Model=Llama-3.1-8B Instruct2026.04 | 47.2 | — | |
| SC-3Model=LLaMA-3.2-3B Instruct2026.04 | 46.8 | — | |
| Greedy-CoTModel=LLaMA-3.2-3B Instruct2026.04 | 46.6 | — |