# Parameter Prediction for Unseen Deep Architectures Source: [Parameter Prediction for Unseen Deep Architectures]() Collection index: [llms.txt]() Data updated: 2026-01-04T03:33:13.123Z Authors: Boris Knyazev, Michal Drozdzal, Graham W. Taylor, Adriana Romero-Soriano Published: 2021-10-25T16:52:33.000Z ## Abstract Deep learning has been successful in automating the design of features in machine learning pipelines. However, the algorithms optimizing neural network parameters remain largely hand-designed and computationally inefficient. We study if we can use deep learning to directly predict these parameters by exploiting the past knowledge of training other networks. We introduce a large-scale dataset of diverse computational graphs of neural architectures - DeepNets-1M - and use it to explore parameter prediction on CIFAR-10 and ImageNet. By leveraging advances in graph neural networks, we propose a hypernetwork that can predict performant parameters in a single forward pass taking a fraction of a second, even on a CPU. The proposed model achieves surprisingly good performance on unseen and diverse networks. For example, it is able to predict all 24 million parameters of a ResNet-50 achieving a 60% accuracy on CIFAR-10. On ImageNet, top-5 accuracy of some of our networks approaches 50%. Our task along with the model and results can potentially lead to a new, more computationally efficient paradigm of training networks. Our model also learns a strong representation of neural architectures enabling their analysis. ## Recorded benchmark results A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. - [Image Classification on ImageNet DEEPNETS-1M (test)](): Image Classification; ImageNet DEEPNETS-1M (test); Avg Top-5 Accuracy: 27.2. - [Image Classification on CIFAR-10 ID-TEST DEEPNETS-1M (test)](): Image Classification; CIFAR-10 ID-TEST DEEPNETS-1M (test); Avg Accuracy: 0.669. - [Image Classification on CIFAR-10 OOD DEEPNETS-1M (test)](): Image Classification; CIFAR-10 OOD DEEPNETS-1M (test); Accuracy (WIDE): 64. - [Image Classification on ImageNet OOD DEEPNETS-1M (test)](): Image Classification; ImageNet OOD DEEPNETS-1M (test); Top-5 Accuracy (RESNET): 5.3. ## Original sources and implementation [Original paper]() [PDF]() [Implementation]()