Logical Reasoning on LOGIQA (Hit@1 %)
56.4Hit@1 (LOGIQA)Mistral 24b-Instruct
Evaluation results
| Method | Links | |
|---|---|---|
| Mistral 24b-InstructType=Single-Agent Baseline, Search=Single-Pass2025.10 | 56.4 | |
| GPT-4o-MiniType=Single-Agent Baseline, Search=Single-Pass2025.10 | 50.1 | |
| MCTS(10,3)+MASPRMAgent=Qwen2.5-1.5B-Instruct, Search=MCTS(10,3), Scorer=MASPRM-7B2025.10 | 45.5 | |
| SBS(5,3)+MASPRMAgent=Qwen2.5-1.5B-Instruct, Search=SBS(5,3), Scorer=MASPRM-7B2025.10 | 45 | |
| Llama3.1 8B-InstructType=Single-Agent Baseline, Search=Single-Pass2025.10 | 42.2 | |
| Llama3.2 3B-InstructType=Single-Agent Baseline, Search=Single-Pass2025.10 | 35.3 | |
| Qwen2.5-1.5B-InstructType=Single-Agent Baseline, Search=Single-Pass2025.10 | 25.8 |