Algorithmic Reasoning on BIG-Bench Hard (Word Sorting & Multi-step Arithmetic Test)
80WS AccuracyStrategyLLM
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| StrategyLLMPrompting strategy=Strategy-based2023.11 | 80 | 86.7 | 83.4 | |
| StrategyLLM-SCPrompting strategy=Strategy-based, Sampling strategy=Self-Consistency2023.11 | 79.3 | 90.7 | 85 | |
| StrategyLLM-ZSPrompting strategy=Strategy-based, Evaluation protocol=Zero-shot2023.11 | 78.7 | 89.3 | 84 | |
| SolutionLLMMethod category=Baseline2023.11 | 74.7 | 55.3 | 65 | |
| SPPrompting strategy=Standard2023.11 | 73.3 | 2 | 37.7 | |
| CoTPrompting strategy=Chain-of-Thought2023.11 | 67.2 | 84 | 75.6 | |
| CoT-SCPrompting strategy=Chain-of-Thought, Sampling strategy=Self-Consistency2023.11 | 61.3 | 86 | 73.7 |