# Geometry-Aware Adaptation for Pretrained Models

Source: [Geometry-Aware Adaptation for Pretrained Models](<https://www.sota2.com/research/paper/68fa71451e209131cea50441>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2026-01-04T04:37:07.271Z

Authors: Nicholas Roberts, Xintong Li, Dyah Adila, Sonia Cromp, Tzu-Heng Huang, Jitian Zhao, Frederic Sala

Published: 2023-07-23T04:48:41.000Z

## Abstract

Machine learning models -- including prominent zero-shot models -- are often trained on datasets whose labels are only a small proportion of a larger label space. Such spaces are commonly equipped with a metric that relates the labels via distances between them. We propose a simple approach to exploit this information to adapt the trained model to reliably predict new classes -- or, in the case of zero-shot prediction, to improve its performance -- without any additional training. Our technique is a drop-in replacement of the standard prediction rule, swapping argmax with the Fr\\'echet mean. We provide a comprehensive theoretical analysis for this approach, studying (i) learning-theoretic results trading off label space diameter, sample complexity, and model dimension, (ii) characterizations of the full range of scenarios in which it is possible to predict any unobserved class, and (iii) an optimal active learning-like next class selection procedure to obtain optimal training classes for when it is not possible to predict the entire range of unobserved classes. Empirically, using easily-available external metrics, our proposed approach, Loki, gains up to 29.7% relative improvement over SimCLR on ImageNet and scales to hundreds of thousands of classes. When no such metric is available, Loki can use self-derived metrics from class embeddings and obtains a 10.5% improvement on pretrained zero-shot models such as CLIP.

## Recorded benchmark results

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

- [Hierarchical Text Classification on LSHTC](<https://www.sota2.com/research/sota/hierarchical-text-classification-on-lshtc.md>): Hierarchical Text Classification; LSHTC; Mean Squared Distance: 0.2896.

- [Text Classification on PubMed 9,419 class subset](<https://www.sota2.com/research/sota/text-classification-on-pubmed-9-419-class-subset.md>): Text Classification; PubMed 9,419 class subset; Mean Squared Distance: 1.42591.

## Original sources and implementation

[Original paper](<https://arxiv.org/abs/2307.12226>)

[PDF](<https://arxiv.org/pdf/2307.12226.pdf>)

[Implementation](<https://github.com/SprocketLab/loki>)
