# Most Ligand-Based Classification Benchmarks Reward Memorization Rather than Generalization

Source: [Most Ligand-Based Classification Benchmarks Reward Memorization Rather than Generalization](<https://www.sota2.com/research/paper/68fa5eda1e209131ce9e5b6f>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2025-10-25T19:01:51.543Z

Authors: Izhar Wallach, Abraham Heifets

Published: 2017-06-20T18:47:46.000Z

## Abstract

Undetected overfitting can occur when there are significant redundancies between training and validation data. We describe AVE, a new measure of training-validation redundancy for ligand-based classification problems that accounts for the similarity amongst inactive molecules as well as active. We investigated seven widely-used benchmarks for virtual screening and classification, and show that the amount of AVE bias strongly correlates with the performance of ligand-based predictive methods irrespective of the predicted property, chemical fingerprint, similarity measure, or previously-applied unbiasing techniques. Therefore, it may be that the previously-reported performance of most ligand-based methods can be explained by overfitting to benchmarks rather than good prospective accuracy.

## Recorded benchmark results

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

- [Virtual Screening on DUD-E 10 diverse targets](<https://www.sota2.com/research/sota/virtual-screening-on-dud-e-10-diverse-targets.md>): Virtual Screening; DUD-E 10 diverse targets; Mean AUC: 99.1.

## Original sources and implementation

[Original paper](<https://arxiv.org/abs/1706.06619>)

[PDF](<https://arxiv.org/pdf/1706.06619.pdf>)