Logical Deduction (Seven Objects) on BBH (test)
55.2AccuracyTextReg
Evaluation Results
| Method | Links | |
|---|---|---|
| TextRegTest Engine=Phi-3.5-Mini-Instruct2026.05 | 55.2 | |
| TextRegTest Engine=Llama-3-8B-Instruct2026.05 | 52 | |
| TextRegTest Engine=Llama-3.1-8B-Instruct2026.05 | 51 | |
| REVOLVETest Engine=Llama-3.1-8B-Instruct2026.05 | 50.6 | |
| CoTTest Engine=Phi-3.5-Mini-Instruct2026.05 | 50.5 | |
| CoTTest Engine=Llama-3.1-8B-Instruct2026.05 | 50.4 | |
| TextGradTest Engine=Llama-3.1-8B-Instruct2026.05 | 50.4 | |
| CoTTest Engine=Llama-3-8B-Instruct2026.05 | 48.8 | |
| REVOLVETest Engine=Llama-3-8B-Instruct2026.05 | 48.8 | |
| TextRegTest Engine=Qwen2-7B-Instruct2026.05 | 47.8 | |
| REVOLVETest Engine=Qwen2-7B-Instruct2026.05 | 47.6 | |
| DLN-1LLM=GPT-32023.06 | 47.5 | |
| CoTTest Engine=Qwen2-7B-Instruct2026.05 | 47.4 | |
| TextGradTest Engine=Qwen2-7B-Instruct2026.05 | 46.6 | |
| TextGradTest Engine=Phi-3.5-Mini-Instruct2026.05 | 46.4 | |
| 0-shotLLM=GPT-32023.06 | 45.9 | |
| DLN-2LLM=GPT-32023.06 | 45.7 | |
| APE-400LLM=GPT-32023.06 | 45.6 | |
| APELLM=GPT-32023.06 | 45.5 | |
| CoTLLM=GPT-32023.06 | 41.1 | |
| TextGradTest Engine=Llama-3-8B-Instruct2026.05 | 41.1 | |
| REVOLVETest Engine=Phi-3.5-Mini-Instruct2026.05 | 38.7 |