Foundation Model or Finetune? Evaluation of few-shot semantic segmentation for river pollution

Foundation models (FMs) are a popular topic of research in AI. Their ability to generalize to new tasks and datasets without retraining or needing an abundance of data makes them an appealing candidate for applications on specialist datasets. In this work, we compare the performance of FMs to finetuned pre-trained supervised models in the task of semantic segmentation on an entirely new dataset. We see that finetuned models consistently outperform the FMs tested, even in cases were data is scarce. We release the code and dataset for this work on GitHub.

Marga Don, Stijn Pinson, Blanca Guillen Cebrian, Yuki M. Asano · 2024