# Task performance prediction on Task-solver pool (held-out)

Source: [Task performance prediction on Task-solver pool (held-out)](<https://www.sota2.com/research/sota/task-performance-prediction-on-task-solver-pool-held-out>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2026-06-09T06:03:49.043Z

Evaluation of model performance predicting task outcomes using the held-out split of the Task-solver pool dataset, measured by AUC and statistical significance metrics.

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

| Scope | Recorded value |
| --- | --- |
| Task | Task performance prediction |
| Dataset (including any stated split) | Task-solver pool (held-out) |
| Metric | AUC |
| Unit | PERCENT |
| Direction | Higher is better |
| Results loaded | 5 of 5 |
| Coverage | All results currently returned for this benchmark |

Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol.

## Metrics

- [AUC](<https://www.sota2.com/research/sota/task-performance-prediction-on-task-solver-pool-held-out.md?metric=AUC>): Higher is better; unit: PERCENT.

- [AUC 95% CI](<https://www.sota2.com/research/sota/task-performance-prediction-on-task-solver-pool-held-out.md?metric=AUC%2095%25%20CI>): Higher is better; unit: SCALAR.

- [Null Mean (Label Permutation)](<https://www.sota2.com/research/sota/task-performance-prediction-on-task-solver-pool-held-out.md?metric=Null%20Mean%20(Label%20Permutation)>): Higher is better; unit: SCALAR.

- [p-value](<https://www.sota2.com/research/sota/task-performance-prediction-on-task-solver-pool-held-out.md?metric=p-value>): Lower is better; unit: SCALAR.

## Recorded results

| Method | AUC | Result date | Report paper |
| --- | --- | --- | --- |
| Structural Grid Descriptors (n\_components\_final) | 80.2 | Not recorded | 6a279e7e90966ade919bfc04 |
| Structural Grid Descriptors (n\_components\_final) | 77.7 | Not recorded | 6a279e7e90966ade919bfc04 |
| Structural Grid Descriptors (n\_components\_final) | 76.5 | Not recorded | 6a279e7e90966ade919bfc04 |
| Structural Grid Descriptors (n\_components\_final) | 69.2 | Not recorded | 6a279e7e90966ade919bfc04 |
| Structural Grid Descriptors (n\_components\_final) | 68.5 | Not recorded | 6a279e7e90966ade919bfc04 |

### Structural Grid Descriptors (n\_components\_final)

AUC: 80.2. Result ID: 6a27acb0fbc373592c239a50.

[Reporting paper](<https://www.sota2.com/research/paper/6a279e7e90966ade919bfc04.md>)

[Method origin paper](<https://www.sota2.com/research/paper/6a279e7e90966ade919bfc04.md>)

Recorded evaluation configuration: {"Solver":"B","Source category":"training","Sample size (n)":"17"}. This record does not assert independent reproduction.

### Structural Grid Descriptors (n\_components\_final)

AUC: 77.7. Result ID: 6a27acb0fbc373592c239a4f.

[Reporting paper](<https://www.sota2.com/research/paper/6a279e7e90966ade919bfc04.md>)

[Method origin paper](<https://www.sota2.com/research/paper/6a279e7e90966ade919bfc04.md>)

Recorded evaluation configuration: {"Solver":"A","Source category":"evaluation","Sample size (n)":"14"}. This record does not assert independent reproduction.

### Structural Grid Descriptors (n\_components\_final)

AUC: 76.5. Result ID: 6a27acb0fbc373592c239a4e.

[Reporting paper](<https://www.sota2.com/research/paper/6a279e7e90966ade919bfc04.md>)

[Method origin paper](<https://www.sota2.com/research/paper/6a279e7e90966ade919bfc04.md>)

Recorded evaluation configuration: {"Sample size (n)":"41","Bootstrap permutations":"2000"}. This record does not assert independent reproduction.

### Structural Grid Descriptors (n\_components\_final)

AUC: 69.2. Result ID: 6a27acb0fbc373592c239a51.

[Reporting paper](<https://www.sota2.com/research/paper/6a279e7e90966ade919bfc04.md>)

[Method origin paper](<https://www.sota2.com/research/paper/6a279e7e90966ade919bfc04.md>)

Recorded evaluation configuration: {"Solver":"A","Source category":"training held-out","Sample size (n)":"4"}. This record does not assert independent reproduction.

### Structural Grid Descriptors (n\_components\_final)

AUC: 68.5. Result ID: 6a27acb0fbc373592c239a52.

[Reporting paper](<https://www.sota2.com/research/paper/6a279e7e90966ade919bfc04.md>)

[Method origin paper](<https://www.sota2.com/research/paper/6a279e7e90966ade919bfc04.md>)

Recorded evaluation configuration: {"Solver":"B","Source category":"evaluation","Sample size (n)":"6"}. This record does not assert independent reproduction.

Page 1 of 1. This is an index page, not the full collection.