# Maximally Useful and Minimally Redundant: The Key to Self Supervised Learning for Imbalanced Data Source: [Maximally Useful and Minimally Redundant: The Key to Self Supervised Learning for Imbalanced Data]() Collection index: [llms.txt]() Data updated: 2026-04-04T05:00:04.515Z Authors: Yash Kumar Sharma, Vineet Padmanabhan Published: 2025-09-10T10:18:20.000Z ## Abstract Contrastive self supervised learning(CSSL) usually makes use of the multi-view assumption which states that all relevant information must be shared between all views. The main objective of CSSL is to maximize the mutual information(MI) between representations of different views and at the same time compress irrelevant information in each representation. Recently, as part of future work, Schwartz Ziv & Yan LeCun pointed out that, when the multi-view assumption is violated, one of the most significant challenges in SSL is in identifying new methods to separate relevant from irrelevant information based on alternative assumptions. Taking a cue from this intuition we make the following contributions in this paper: 1) We develop a CSSL framework wherein multiple images and multiple views(MIMV) are considered as input, which is different from the traditional multi-view assumption 2) We adopt a novel augmentation strategy that includes both normalized (invertible) and augmented (non-invertible) views so that complete information of one image can be preserved and hard augmentation can be chosen for the other image 3) An Information bottleneck(IB) principle is outlined for MIMV to produce optimal representations 4) We introduce a loss function that helps to learn better representations by filtering out extreme features 5) The robustness of our proposed framework is established by applying it to the imbalanced dataset problem wherein we achieve a new state-of-the-art accuracy (2% improvement in Cifar10-LT using Resnet-18, 5% improvement in Cifar100-LT using Resnet-18 and 3% improvement in Imagenet-LT (1k) using Resnet-50). ## Recorded benchmark results A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. - [Image Classification on Cifar100-LT exponential](): Image Classification; Cifar100-LT exponential; Accuracy: 60.18. - [Image Classification on Cifar10-LT exponential](): Image Classification; Cifar10-LT exponential; Accuracy: 83.66. - [Image Classification on ImageNet LT (Sampling Factor Evaluation)](): Image Classification; ImageNet LT; Top-1 Accuracy (s=1.0): 52.9. ## Original sources and implementation [Original paper]() [PDF]()