# Prompt Optimization Efficiency on BBH, GSM8K, and CSQA (test)

Source: [Prompt Optimization Efficiency on BBH, GSM8K, and CSQA (test)](<https://www.sota2.com/research/sota/prompt-optimization-efficiency-on-bbh-gsm8k-and-csqa-test>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2026-03-18T08:01:44.272Z

Evaluation of the efficiency of prompt optimization techniques, measured by optimization time, added tokens, and LLM calls, averaged across the test splits of Big-Bench Hard (BBH), GSM8K, and CSQA datasets.

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

| Scope | Recorded value |
| --- | --- |
| Task | Prompt Optimization Efficiency |
| Dataset (including any stated split) | BBH, GSM8K, and CSQA averaged (test) |
| Metric | Opt. Time (s) |
| Unit | TIME |
| Direction | Lower is better |
| Results loaded | 5 of 5 |
| Coverage | All results currently returned for this benchmark |

Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol.

## Metrics

- [Opt. Time (s)](<https://www.sota2.com/research/sota/prompt-optimization-efficiency-on-bbh-gsm8k-and-csqa-test.md?metric=Opt.%20Time%20(s)>): Lower is better; unit: TIME.

- [Added Tokens](<https://www.sota2.com/research/sota/prompt-optimization-efficiency-on-bbh-gsm8k-and-csqa-test.md?metric=Added%20Tokens>): Lower is better; unit: SCALAR.

- [LLM Calls](<https://www.sota2.com/research/sota/prompt-optimization-efficiency-on-bbh-gsm8k-and-csqa-test.md?metric=LLM%20Calls>): Lower is better; unit: SCALAR.

## Recorded results

| Method | Opt. Time (s) | Result date | Report paper |
| --- | --- | --- | --- |
| No Optimization | 0 | Not recorded | 69ba31a80636e5bf4afc096d |
| POaaS | 0.95 | Not recorded | 69ba31a80636e5bf4afc096d |
| PromptWizard | 336 | Not recorded | 69ba31a80636e5bf4afc096d |
| OPRO | 455 | Not recorded | 69ba31a80636e5bf4afc096d |
| EvoPrompt | 602 | Not recorded | 69ba31a80636e5bf4afc096d |

### No Optimization

Opt. Time (s): 0. Result ID: 69ba5ba347f34ce924a40a44.

[Reporting paper](<https://www.sota2.com/research/paper/69ba31a80636e5bf4afc096d.md>)

Recorded evaluation configuration: {"Model":"Llama-3.2-3B"}. This record does not assert independent reproduction.

### POaaS

Opt. Time (s): 0.95. Result ID: 69ba5ba347f34ce924a40a48.

[Reporting paper](<https://www.sota2.com/research/paper/69ba31a80636e5bf4afc096d.md>)

[Method origin paper](<https://www.sota2.com/research/paper/69ba31a80636e5bf4afc096d.md>)

Recorded evaluation configuration: {"Model":"Llama-3.2-3B"}. This record does not assert independent reproduction.

### PromptWizard

Opt. Time (s): 336. Result ID: 69ba5ba347f34ce924a40a46.

[Reporting paper](<https://www.sota2.com/research/paper/69ba31a80636e5bf4afc096d.md>)

Recorded evaluation configuration: {"Model":"Llama-3.2-3B"}. This record does not assert independent reproduction.

### OPRO

Opt. Time (s): 455. Result ID: 69ba5ba347f34ce924a40a45.

[Reporting paper](<https://www.sota2.com/research/paper/69ba31a80636e5bf4afc096d.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa72f31e209131cea597cc.md>)

Recorded evaluation configuration: {"Model":"Llama-3.2-3B"}. This record does not assert independent reproduction.

### EvoPrompt

Opt. Time (s): 602. Result ID: 69ba5ba347f34ce924a40a47.

[Reporting paper](<https://www.sota2.com/research/paper/69ba31a80636e5bf4afc096d.md>)

Recorded evaluation configuration: {"Model":"Llama-3.2-3B"}. This record does not assert independent reproduction.

Page 1 of 1. This is an index page, not the full collection.
