# Semantically-Conditioned Negative Samples for Efficient Contrastive Learning Source: [Semantically-Conditioned Negative Samples for Efficient Contrastive Learning]() Collection index: [llms.txt]() Data updated: 2025-10-25T22:57:51.173Z Authors: James O' Neill, Danushka Bollegala Published: 2021-02-12T16:26:52.000Z ## Abstract Negative sampling is a limiting factor w.r.t. the generalization of metric-learned neural networks. We show that uniform negative sampling provides little information about the class boundaries and thus propose three novel techniques for efficient negative sampling: drawing negative samples from (1) the top-$k$ most semantically similar classes, (2) the top-$k$ most semantically similar samples and (3) interpolating between contrastive latent representations to create pseudo negatives. Our experiments on CIFAR-10, CIFAR-100 and Tiny-ImageNet-200 show that our proposed \\textit{Semantically Conditioned Negative Sampling} and Latent Mixup lead to consistent performance improvements. In the standard supervised learning setting, on average we increase test accuracy by 1.52\\% percentage points on CIFAR-10 across various network architectures. In the knowledge distillation setting, (1) the performance of student networks increase by 4.56\\% percentage points on Tiny-ImageNet-200 and 3.29\\% on CIFAR-100 over student networks trained with no teacher and (2) 1.23\\% and 1.72\\% respectively over a \\textit{hard-to-beat} baseline (Hinton et al., 2015). ## Recorded benchmark results A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. ## Original sources and implementation [Original paper]() [PDF]()