Logical Reasoning on PrOntoQA 5-hop
90.2AccuracyLSS-CoT
Evaluation results
| Method | Links | ||
|---|---|---|---|
| LSS-CoTModel=Gemma-2-9B It2026.04 | 90.2 | 2.8 | |
| SC-3Model=Gemma-2-9B It2026.04 | 90 | — | |
| Greedy-CoTModel=Gemma-2-9B It2026.04 | 87.4 | — | |
| 3-shot-CoTModel=Gemma-2-9B It2026.04 | 82.8 | — | |
| SC-3Model=Llama-3.1-8B Instruct2026.04 | 79 | — | |
| LSS-CoTModel=Llama-3.1-8B Instruct2026.04 | 75.4 | 4.8 | |
| 3-shot-CoTModel=Llama-3.1-8B Instruct2026.04 | 72.4 | — | |
| Greedy-CoTModel=Llama-3.1-8B Instruct2026.04 | 70.6 | — | |
| LSS-CoTModel=Phi-3-mini-4k Instruct2026.04 | 70.6 | 11 | |
| SC-3Model=Phi-3-mini-4k Instruct2026.04 | 70.2 | — | |
| 3-shot-CoTModel=Phi-3-mini-4k Instruct2026.04 | 63.8 | — | |
| 3-shot-CoTModel=LLaMA-3.2-3B Instruct2026.04 | 61.4 | — | |
| Greedy-CoTModel=Phi-3-mini-4k Instruct2026.04 | 59.6 | — | |
| SC-3Model=LLaMA-2-13B Chat2026.04 | 55.4 | — | |
| LSS-CoTModel=LLaMA-2-13B Chat2026.04 | 55 | 6.4 | |
| LSS-CoTModel=LLaMA-3.2-3B Instruct2026.04 | 53.4 | 3.4 | |
| 3-shot-CoTModel=LLaMA-2-13B Chat2026.04 | 51.2 | — | |
| Greedy-CoTModel=LLaMA-3.2-3B Instruct2026.04 | 50 | — | |
| SC-3Model=LLaMA-3.2-3B Instruct2026.04 | 49.4 | — | |
| Greedy-CoTModel=LLaMA-2-13B Chat2026.04 | 48.6 | — |