# Test-time routing on 10-cell Steerable-Hard (evaluation set)

Source: [Test-time routing on 10-cell Steerable-Hard (evaluation set)](<https://www.sota2.com/research/sota/test-time-routing-on-10-cell-steerable-hard-evaluation-set>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2026-06-04T13:35:14.793Z

Evaluation of Test-time routing performance on the evaluation set of the 10-cell Steerable-Hard dataset, measured by rescue rate and retry delta.

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

| Scope | Recorded value |
| --- | --- |
| Task | Test-time routing |
| Dataset (including any stated split) | 10-cell Steerable-Hard (evaluation set) |
| Metric | Rescue Rate |
| Unit | PERCENT |
| Direction | Higher is better |
| Results loaded | 5 of 5 |
| Coverage | All results currently returned for this benchmark |

Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol.

## Metrics

- [Rescue Rate](<https://www.sota2.com/research/sota/test-time-routing-on-10-cell-steerable-hard-evaluation-set.md?metric=Rescue%20Rate>): Higher is better; unit: PERCENT.

- [Retry Delta](<https://www.sota2.com/research/sota/test-time-routing-on-10-cell-steerable-hard-evaluation-set.md?metric=Retry%20Delta>): Lower is better; unit: SCALAR.

## Recorded results

| Method | Rescue Rate | Result date | Report paper |
| --- | --- | --- | --- |
| ProblemOracle | 46.8 | Not recorded | 6a210709c10484bac23f0837 |
| DiagnosticOracle | 32.1 | Not recorded | 6a210709c10484bac23f0837 |
| Outcome-trained | 30.2 | Not recorded | 6a210709c10484bac23f0837 |
| Feature-only | 28.5 | Not recorded | 6a210709c10484bac23f0837 |
| Retry | 16.3 | Not recorded | 6a210709c10484bac23f0837 |

### ProblemOracle

Rescue Rate: 46.8. Result ID: 6a217f0bfbc373592c1f05c1.

[Reporting paper](<https://www.sota2.com/research/paper/6a210709c10484bac23f0837.md>)

Recorded evaluation configuration: {"Routing info":"per problem-unit arg max over operators (not deployable)","Needs labels":"yes","Compute (x retry)":"66.5"}. This record does not assert independent reproduction.

### DiagnosticOracle

Rescue Rate: 32.1. Result ID: 6a217f0bfbc373592c1f05c0.

[Reporting paper](<https://www.sota2.com/research/paper/6a210709c10484bac23f0837.md>)

Recorded evaluation configuration: {"Routing info":"runs every operator on the test cell, picks modal-best","Needs labels":"yes","Compute (x retry)":"66.5"}. This record does not assert independent reproduction.

### Outcome-trained

Rescue Rate: 30.2. Result ID: 6a217f0bfbc373592c1f05bf.

[Reporting paper](<https://www.sota2.com/research/paper/6a210709c10484bac23f0837.md>)

Recorded evaluation configuration: {"Routing info":"RF classifier on the cell’s own per-pid operator outcomes (assay cost = DiagnosticOracle’s)","Needs labels":"yes","Compute (x retry)":"66.5"}. This record does not assert independent reproduction.

### Feature-only

Rescue Rate: 28.5. Result ID: 6a217f0bfbc373592c1f05be.

[Reporting paper](<https://www.sota2.com/research/paper/6a210709c10484bac23f0837.md>)

[Method origin paper](<https://www.sota2.com/research/paper/6a210709c10484bac23f0837.md>)

Recorded evaluation configuration: {"Routing info":"winner-take-all over 3 trajectory features; no training","Needs labels":"no","Compute (x retry)":"1.4"}. This record does not assert independent reproduction.

### Retry

Rescue Rate: 16.3. Result ID: 6a217f0bfbc373592c1f05bd.

[Reporting paper](<https://www.sota2.com/research/paper/6a210709c10484bac23f0837.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa6a5f1e209131cea27ced.md>)

[Implementation](<https://www.github.com/openai/human-eval>)

Recorded evaluation configuration: {"Routing info":"no information, no test-cell compute beyond resampling","Needs labels":"no","Compute (x retry)":"1.0"}. This record does not assert independent reproduction.

Page 1 of 1. This is an index page, not the full collection.