Abductive Logical Reasoning on QUAIL
84Accuracy (QUAIL)PACS Llama 3.3 70B
Evaluation results
| Method | Links | |
|---|---|---|
| PACS Llama 3.3 70BMethod=PACS, Model=Llama 3.3, Parameters=70B, Model Type=non-thinking model2026.05 | 84 | |
| PACSBackbone=Llama 3.3 70B2026.05 | 84 | |
| If-BeamBackbone=Llama 3.3 70B2026.05 | 82 | |
| SC OSS-GPT-120BMethod=SC, Model=OSS-GPT, Parameters=120B, Model Type=Thinking model2026.05 | 80 | |
| PACS Llama 3 8BMethod=PACS, Model=Llama 3, Parameters=8B, Model Type=non-thinking model2026.05 | 80 | |
| PACSBackbone=Llama 3-Instruct 8B2026.05 | 80 | |
| ARGOSBackbone=Llama 3.3 70B2026.05 | 80 | |
| SC-20Backbone=Llama 3.3 70B2026.05 | 75 | |
| If-BeamBackbone=Llama 3-Instruct 8B2026.05 | 74 | |
| ARGOSBackbone=Llama 3-Instruct 8B2026.05 | 73 | |
| LoTBackbone=Llama 3.3 70B2026.05 | 72 | |
| COTBackbone=Llama 3.3 70B2026.05 | 72 | |
| SC-20Backbone=Llama 3-Instruct 8B2026.05 | 70 | |
| COTBackbone=Llama 3-Instruct 8B2026.05 | 66 | |
| LoTBackbone=Llama 3-Instruct 8B2026.05 | 56 | |
| LLM-TreeBackbone=Llama 3-Instruct 8B2026.05 | 56 | |
| LLM-TreeBackbone=Llama 3.3 70B2026.05 | 56 |