Hyperparameter Optimization on autoresearch (val)
99.04Best Validation bpbKarpathy Agent (14 HPs)
Evaluation results
| Method | Links | ||||||||
|---|---|---|---|---|---|---|---|---|---|
| Karpathy Agent (14 HPs)LLM variant=27B, Number of seeds=32026.03 | 99.04 | 289 | 1 | 0.035 | 0.225 | 0.057 | 29 | — | |
| LLAMBO (Optuna)LLM variant=27B, Number of seeds=32026.03 | 98.82 | 629 | 61 | 0.251 | 1.247 | 1.13 | 458 | — | |
| RandomNumber of seeds=32026.03 | 98.73 | 568 | 56 | 0.273 | 1.38 | 1.382 | 514 | — | |
| LLAMBO (Paper)LLM variant=27B, Number of seeds=32026.03 | 98.62 | 496 | 48 | 0.252 | 1.257 | 1.218 | 364 | — | |
| Karpathy Agent (Code)LLM variant=27B, Number of seeds=32026.03 | 98.14 | 324 | 12 | — | — | — | — | — | |
| CMA-ESNumber of seeds=32026.03 | 97.85 | 336 | 16 | 0.158 | 0.794 | 0.58 | 802 | — | |
| SMACNumber of seeds=32026.03 | 97.78 | 431 | 36 | 0.238 | 1.191 | 0.369 | 224 | — | |
| TPENumber of seeds=32026.03 | 97.68 | 317 | 11 | 0.196 | 0.996 | 0.413 | 518 | — | |
| CentaurLLM variant=27B, Number of seeds=32026.03 | 97.63 | 334 | 15 | 0.115 | 0.55 | 0.341 | 330 | — | |
| Model Aautonomous experiments=123, seeds=32026.06 | 0.9822 | — | — | — | — | — | — | 0.9799 | |
| Model Bautonomous experiments=123, seeds=32026.06 | 0.9793 | — | — | — | — | — | — | 0.9758 | |
| Model Cautonomous experiments=123, seeds=32026.06 | 0.9781 | — | — | — | — | — | — | 0.9766 | |
| Fugu-Ultraautonomous experiments=123, seeds=32026.06 | 0.9774 | — | — | — | — | — | — | 0.9748 |