# Test-time routing on 10-cell (failed problem instances)

Source: [Test-time routing on 10-cell (failed problem instances)](<https://www.sota2.com/research/sota/test-time-routing-on-10-cell-failed-problem-instances>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2026-06-04T13:35:14.766Z

Evaluation of Test-time routing performance on the 10-cell (failed problem instances) dataset, measured by rescue rate and delta retry.

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

| Scope | Recorded value |
| --- | --- |
| Task | Test-time routing |
| Dataset (including any stated split) | 10-cell (failed problem instances) |
| Metric | Rescue Rate |
| Unit | PERCENT |
| Direction | Higher is better |
| Results loaded | 5 of 5 |
| Coverage | All results currently returned for this benchmark |

Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol.

## Metrics

- [Rescue Rate](<https://www.sota2.com/research/sota/test-time-routing-on-10-cell-failed-problem-instances.md?metric=Rescue%20Rate>): Higher is better; unit: PERCENT.

- [Delta Retry](<https://www.sota2.com/research/sota/test-time-routing-on-10-cell-failed-problem-instances.md?metric=Delta%20Retry>): Lower is better; unit: SCALAR.

## Recorded results

| Method | Rescue Rate | Result date | Report paper |
| --- | --- | --- | --- |
| ProblemOracle | 51.4 | Not recorded | 6a210709c10484bac23f0837 |
| DiagnosticOracle | 39 | Not recorded | 6a210709c10484bac23f0837 |
| Outcome-trained | 37.4 | Not recorded | 6a210709c10484bac23f0837 |
| Feature-only | 37.3 | Not recorded | 6a210709c10484bac23f0837 |
| Retry | 37.2 | Not recorded | 6a210709c10484bac23f0837 |

### ProblemOracle

Rescue Rate: 51.4. Result ID: 6a217f0bfbc373592c1f05bc.

[Reporting paper](<https://www.sota2.com/research/paper/6a210709c10484bac23f0837.md>)

Recorded evaluation configuration: {"Routing info":"per problem-unit arg max over operators (not deployable)","Needs labels":"yes","Compute (x retry)":"66.5"}. This record does not assert independent reproduction.

### DiagnosticOracle

Rescue Rate: 39. Result ID: 6a217f0bfbc373592c1f05bb.

[Reporting paper](<https://www.sota2.com/research/paper/6a210709c10484bac23f0837.md>)

Recorded evaluation configuration: {"Routing info":"runs every operator on the test cell, picks modal-best","Needs labels":"yes","Compute (x retry)":"66.5"}. This record does not assert independent reproduction.

### Outcome-trained

Rescue Rate: 37.4. Result ID: 6a217f0bfbc373592c1f05ba.

[Reporting paper](<https://www.sota2.com/research/paper/6a210709c10484bac23f0837.md>)

Recorded evaluation configuration: {"Routing info":"RF classifier on the cell’s own per-pid operator outcomes (assay cost = DiagnosticOracle’s)","Needs labels":"yes","Compute (x retry)":"66.5"}. This record does not assert independent reproduction.

### Feature-only

Rescue Rate: 37.3. Result ID: 6a217f0bfbc373592c1f05b9.

[Reporting paper](<https://www.sota2.com/research/paper/6a210709c10484bac23f0837.md>)

[Method origin paper](<https://www.sota2.com/research/paper/6a210709c10484bac23f0837.md>)

Recorded evaluation configuration: {"Routing info":"winner-take-all over 3 trajectory features; no training","Needs labels":"no","Compute (x retry)":"1.4"}. This record does not assert independent reproduction.

### Retry

Rescue Rate: 37.2. Result ID: 6a217f0bfbc373592c1f05b8.

[Reporting paper](<https://www.sota2.com/research/paper/6a210709c10484bac23f0837.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa6a5f1e209131cea27ced.md>)

[Implementation](<https://www.github.com/openai/human-eval>)

Recorded evaluation configuration: {"Routing info":"no information, no test-cell compute beyond resampling","Needs labels":"no","Compute (x retry)":"1.0"}. This record does not assert independent reproduction.

Page 1 of 1. This is an index page, not the full collection.