Logical Reasoning on BigBench Hard Boolean Expressions
76.8AccuracyReElicit
Evaluation results
| Method | Links | |
|---|---|---|
| ReElicitevaluations=30 prompt evaluations2026.05 | 76.8 | |
| DARTBase Model=Qwen3-0.6B2026.05 | 73.9 | |
| GRPOBase Model=Qwen3-0.6B2026.05 | 73.7 | |
| STaRBase Model=Qwen3-0.6B2026.05 | 73 | |
| TESSYBase Model=Qwen3-0.6B2026.05 | 72.8 | |
| BaseBase Model=Qwen3-0.6B2026.05 | 72.6 | |
| APEevaluations=30 prompt evaluations2026.05 | 71.9 | |
| TextGradevaluations=30 prompt evaluations2026.05 | 66.9 | |
| OPROevaluations=30 prompt evaluations2026.05 | 65 | |
| PromptBreederevaluations=30 prompt evaluations2026.05 | 64.3 | |
| Original-SFTBase Model=Qwen3-0.6B2026.05 | 63 | |
| Original-SFTBase Model=Qwen2.5-0.5B-Instruct2026.05 | 56.2 | |
| GRPOBase Model=Qwen2.5-0.5B-Instruct2026.05 | 50.3 | |
| DARTBase Model=Qwen2.5-0.5B-Instruct2026.05 | 50.1 | |
| STaRBase Model=Qwen2.5-0.5B-Instruct2026.05 | 47.1 | |
| BaseBase Model=Qwen2.5-0.5B-Instruct2026.05 | 46.6 | |
| TESSYBase Model=Qwen2.5-0.5B-Instruct2026.05 | 39.4 |