Logical Deduction (5 Objects) on Test Set
61.1AccuracyTextReg
Evaluation results
| Method | Links | |
|---|---|---|
| TextRegTest Engine=Llama-3.1-8B-Instruct2026.05 | 61.1 | |
| TextGradTest Engine=Llama-3.1-8B-Instruct2026.05 | 59.8 | |
| CoTTest Engine=Llama-3.1-8B-Instruct2026.05 | 59.7 | |
| TextRegTest Engine=Phi-3.5-Mini-Instruct2026.05 | 57.9 | |
| REVOLVETest Engine=Llama-3.1-8B-Instruct2026.05 | 57.7 | |
| CoTTest Engine=Phi-3.5-Mini-Instruct2026.05 | 57 | |
| TextRegTest Engine=Qwen2-7B-Instruct2026.05 | 55.3 | |
| REVOLVETest Engine=Qwen2-7B-Instruct2026.05 | 54.4 | |
| TextRegTest Engine=Llama-3-8B-Instruct2026.05 | 53.3 | |
| CoTTest Engine=Llama-3-8B-Instruct2026.05 | 52.6 | |
| CoTTest Engine=Qwen2-7B-Instruct2026.05 | 51.6 | |
| REVOLVETest Engine=Llama-3-8B-Instruct2026.05 | 51.5 | |
| TextGradTest Engine=Qwen2-7B-Instruct2026.05 | 51.3 | |
| TextGradTest Engine=Phi-3.5-Mini-Instruct2026.05 | 46.1 | |
| REVOLVETest Engine=Phi-3.5-Mini-Instruct2026.05 | 43.4 | |
| TextGradTest Engine=Llama-3-8B-Instruct2026.05 | 42.1 |