Logical Reasoning on PrOntoQA
91.4Calibrated AccuracyLlama 3.1 8B
Evaluation results
| Method | Links | |
|---|---|---|
| Llama 3.1 8BModel Size=8B2026.05 | 91.4 | |
| Qwen2.5-7BModel Size=7B2026.05 | 80.7 | |
| Gemma 3 27BModel Size=27B2026.05 | 77.4 | |
| DS-R1-7BModel Size=7B2026.05 | 77.1 | |
| Qwen2.5-32BModel Size=32B2026.05 | 67.8 | |
| SC+IC (tune)Backbone=MIXTRAL-8×7B, Prompting Strategy=CoT2024.05 | 63.8 | |
| Qwen 3.5 9BModel Size=9B2026.05 | 63.5 | |
| SC+IC (tune)Backbone=MISTRAL-7B, Prompting Strategy=CoT2024.05 | 60.4 | |
| SC+IC (tune)Backbone=MIXTRAL-8×7B, Prompting Strategy=L2M2024.05 | 59.3 | |
| SC+IC (tune)Backbone=LLAMA-2-13B, Prompting Strategy=L2M2024.05 | 56.6 | |
| SC+IC (tune)Backbone=MISTRAL-7B, Prompting Strategy=L2M2024.05 | 56.6 | |
| SC+IC (tune)Backbone=LLAMA-2-7B, Prompting Strategy=L2M2024.05 | 55.7 | |
| SC+IC (tune)Backbone=LLAMA-2-13B, Prompting Strategy=CoT2024.05 | 54.5 | |
| DS-R1-32BModel Size=32B2026.05 | 52.9 | |
| SC+IC (tune)Backbone=LLAMA-2-7B, Prompting Strategy=CoT2024.05 | 50.8 | |
| Qwen2.5-14BModel Size=14B2026.05 | 49.6 | |
| DS-R1-14BModel Size=14B2026.05 | 49.5 |