Performance Prediction on BIG-bench (train)
2.487Boolean Expressions ScoreSize
Evaluation Results
| Method | Links | ||||||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| SizeDescription=Logistic scaling laws using model parameter size (Owen, 2024)2026.02 | 2.487 | 0.846 | 1.419 | 2.592 | 1.49 | 1.891 | 1.921 | 0.945 | 0.964 | 2.617 | 1.336 | 1.455 | 1.241 | 2.008 | 2.106 | 2.433 | 3.068 | 0.907 | 0.556 | 1.699 | |
| OSLDescription=PCA decomposition approach from Ruan et al. (2024)2026.02 | 1.153 | 0.469 | 0.462 | 1.519 | 0.984 | 0.965 | 0.993 | 0.156 | 0.362 | 2.034 | 0.737 | 0.435 | 0.331 | 1.169 | 1.274 | 1.197 | 1.746 | 0.717 | 0.528 | 0.907 | |
| SCDescription=Structured capabilities model with latent factor capability estimates2026.02 | 1.056 | 0.476 | 0.464 | 1.536 | 0.961 | 0.937 | 0.994 | 0.147 | 0.348 | 1.877 | 0.707 | 0.423 | 0.324 | 0.902 | 1.248 | 1.099 | 1.691 | 0.681 | 0.477 | 0.86 |