# REMSA: Foundation Model Selection for Remote Sensing via a Constraint-Aware Agent

Source: [REMSA: Foundation Model Selection for Remote Sensing via a Constraint-Aware Agent](<https://www.sota2.com/research/paper/6923e67dd6b0dd334292ef85>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2026-07-02T05:00:05.659Z

Authors: Binger Chen, Tacettin Emre B\\"ok, Behnood Rasti, Volker Markl, Beg\\"um Demir

Published: 2025-11-21T17:41:26.000Z

## Abstract

Foundation Models (FMs) are increasingly integrated into remote sensing (RS) pipelines. These models include unimodal vision encoders and multimodal architectures. FMs are adapted to diverse perception tasks, such as image classification, change detection, and visual question answering. However, selecting the most suitable remote sensing foundation model (RSFM) for a specific task remains challenging due to scattered documentation, heterogeneous formats, and complex deployment constraints. To address this, we first introduce the RSFM Database (RS-FMD), the first structured and schema-guided resource covering over 160 RSFMs trained on various data modalities, spanning different spatial, spectral, and temporal resolutions, considering different learning paradigms. Built upon RS-FMD, we further present REMSA, a constraint-aware agent that enables automated RSFM selection from natural language queries. REMSA combines structured FM metadata retrieval with a task-driven decision workflow. In detail, it interprets user input, clarifies missing constraints, ranks models via in-context learning, and provides transparent justifications. Our system supports various RS tasks and data modalities, enabling personalized, reproducible, and efficient FM selection. To evaluate REMSA, we construct a benchmark of 100 expert-verified RS query scenarios. Each query is evaluated across 4 systems and 3 LLM backbones, with the top-3 selected models manually assessed by domain experts. This results in 3,000 expert-scored task--system--model configurations under our novel expert-centered evaluation protocol. REMSA outperforms multiple baselines, showing its practical utility in real decision-making applications. REMSA operates entirely on publicly available metadata of open source RSFMs, without accessing private or sensitive data.

## Recorded benchmark results

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

- [Foundation Model Selection on 100 diverse natural language user queries (test)](<https://www.sota2.com/research/sota/foundation-model-selection-on-100-diverse-natural-language-user-queries-test.md>): Foundation Model Selection; 100 diverse natural language user queries (test); Average Top-1: 75.76.

- [Foundation Model Selection on RS Queries](<https://www.sota2.com/research/sota/foundation-model-selection-on-rs-queries.md>): Foundation Model Selection; RS Queries; Avg Top-1: 75.35.

- [Foundation Model Selection on Remote Sensing Model Selection dataset 100 RS queries](<https://www.sota2.com/research/sota/foundation-model-selection-on-remote-sensing-model-selection-dataset-100-rs-queries.md>): Foundation Model Selection; Remote Sensing Model Selection dataset 100 RS queries; Avg Top-1: 73.39.

## Original sources and implementation

[Original paper](<https://arxiv.org/abs/2511.17442>)

[PDF](<https://arxiv.org/pdf/2511.17442.pdf>)
