Performance Prediction on ARC 1.2k (test)
1.14MAEMetabench
Evaluation results
| Method | Links | ||
|---|---|---|---|
| MetabenchSelection=Best for val., Prediction=ability-IRT2025.10 | 1.14 | 0.971 | |
| DISCOSelection=High PDS, Prediction=Sig. + RF2025.10 | 1.47 | 0.971 | |
| Model signatureSelection=Random, Prediction=Sig. + RF2025.10 | 1.72 | 0.938 | |
| DISCOSelection=High JSD, Prediction=Sig. + RF2025.10 | 1.75 | 0.938 | |
| DISCOSelection=High PDS, Prediction=Sig. + kNN2025.10 | 1.96 | 0.937 | |
| DISCOSelection=High JSD, Prediction=Sig. + kNN2025.10 | 2.11 | 0.939 | |
| tinyBenchmarksSelection=Anchor-corr, Prediction=gp-IRT2025.10 | 2.18 | 0.948 | |
| tinyBenchmarksSelection=Random, Prediction=gp-IRT2025.10 | 2.22 | 0.921 | |
| Model signatureSelection=Random, Prediction=Sig. + kNN2025.10 | 2.3 | 0.905 | |
| BaselineSelection=Random, Prediction=Direct eval.2025.10 | 2.61 | 0.898 | |
| tinyBenchmarksSelection=Anchor-IRT, Prediction=gp-IRT2025.10 | 4.55 | 0.708 |