Symbolic and Logical Reasoning on Big-Bench Hard (BBH)
88.1Exact Match PerformancePromptWizard
Evaluation results
| Method | Links | ||
|---|---|---|---|
| PromptWizardTask Model=GPT-4, Optimizer Model=GPT-42026.02 | 88.1 | — | |
| C-MOP (Ours)Task Model=Q30BA3B-Inst, Optimizer Model=Q30BA3B-Think2026.02 | 86.24 | 82.08 | |
| ERMTask Model=Doubao-Pro, Optimizer Model=GPT-4o2026.02 | 86.1 | — | |
| PromptWizard*Task Model=Q30BA3B-Inst, Optimizer Model=Q30BA3B-Think2026.02 | 85.34 | 80.5 | |
| SPO*Task Model=Q30BA3B-Inst, Optimizer Model=Q30BA3B-Think2026.02 | 84.13 | 78.84 | |
| ProTeGi*Task Model=Q30BA3B-Inst, Optimizer Model=Q30BA3B-Think2026.02 | 82.63 | 78.72 | |
| GPO*Task Model=Q30BA3B-Inst, Optimizer Model=Q30BA3B-Think2026.02 | 82.61 | 78.5 | |
| Zero-shot (Base)Task Model=Q30BA3B-Inst2026.02 | 81.65 | 67.75 | |
| MARSTask Model=GPT-4o, Optimizer Model=GPT-4o2026.02 | 80.86 | — | |
| GPOTask Model=GPT-4, Optimizer Model=GPT-42026.02 | 78.65 | — | |
| EvoPromptTask Model=GPT-3.5-Turbo, Optimizer Model=GPT-3.5-Turbo2026.02 | 75.03 | — | |
| GPT-4Setting=Zero-shot2023.11 | 69.04 | — | |
| ChatGPTSetting=Zero-shot2023.11 | 55.38 | — | |
| WizardLM-70BSetting=Zero-shot2023.11 | 51.08 | — | |
| Orca 2-13BSetting=Zero-shot, System Message=Default2023.11 | 50.18 | — | |
| Orca 2-13BSetting=Zero-shot, System Message=Cautious2023.11 | 50.01 | — | |
| Orca-1-13BSetting=Zero-shot2023.11 | 47.84 | — | |
| Orca 2-7BSetting=Zero-shot, System Message=Default2023.11 | 45.93 | — | |
| LLaMA-2-Chat-70BSetting=Zero-shot2023.11 | 44.68 | — | |
| Orca 2-7BSetting=Zero-shot, System Message=Cautious2023.11 | 42.8 | — | |
| WizardLM-13BSetting=Zero-shot2023.11 | 38.47 | — | |
| LLaMA-2-Chat-13BSetting=Zero-shot2023.11 | 33.6 | — |