# Leveraging Random Label Memorization for Unsupervised Pre-Training Source: [Leveraging Random Label Memorization for Unsupervised Pre-Training]() Collection index: [llms.txt]() Data updated: 2025-10-28T13:56:37.290Z Authors: Vinaychandran Pondenkandath, Michele Alberti, Sammer Puran, Rolf Ingold, Marcus Liwicki Published: 2018-11-05T12:27:14.000Z ## Abstract We present a novel approach to leverage large unlabeled datasets by pre-training state-of-the-art deep neural networks on randomly-labeled datasets. Specifically, we train the neural networks to memorize arbitrary labels for all the samples in a dataset and use these pre-trained networks as a starting point for regular supervised learning. Our assumption is that the "memorization infrastructure" learned by the network during the random-label training proves to be beneficial for the conventional supervised learning as well. We test the effectiveness of our pre-training on several video action recognition datasets (HMDB51, UCF101, Kinetics) by comparing the results of the same network with and without the random label pre-training. Our approach yields an improvement - ranging from 1.5% on UCF-101 to 5% on Kinetics - in classification accuracy, which calls for further research in this direction. ## Recorded benchmark results A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. ## Original sources and implementation [Original paper]() [PDF]()