LLM Workflow Optimization on Big-Bench Hard (test)
78.6BBH Overall AccuracyTrace
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| TraceEvaluation Protocol=0-shot, Chain-of-Thought=true2024.06 | 78.6 | 75.8 | 80.6 | |
| DSPy-POEvaluation Protocol=0-shot, Chain-of-Thought=true, Optimizer=COPRO2024.06 | 71.6 | 73.9 | 70 | |
| DSPyEvaluation Protocol=0-shot, Chain-of-Thought=true2024.06 | 70.4 | 73.7 | 68 | |
| TraceEvaluation Protocol=0-shot, Chain-of-Thought=false2024.06 | 59.5 | 70.9 | 51.1 | |
| DSPy-POEvaluation Protocol=0-shot, Chain-of-Thought=false, Optimizer=COPRO2024.06 | 55.3 | 69 | 45.2 | |
| DSPyEvaluation Protocol=0-shot, Chain-of-Thought=false2024.06 | 41.6 | 53.8 | 32.6 |