# Open-ended Visual Question Answering on EarthVLSet 1.0 (test) Source: [Open-ended Visual Question Answering on EarthVLSet 1.0 (test)]() Collection index: [llms.txt]() Data updated: 2026-02-26T05:37:14.512Z Evaluation of model performance on the test split of the EarthVLSet 1.0 dataset for open-ended visual question answering, typically assessed using generative metrics (BLEU, ROUGE, METEOR, CIDEr) and human judgment. A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. | Scope | Recorded value | | --- | --- | | Task | Open-ended Visual Question Answering | | Dataset (including any stated split) | EarthVLSet 1.0 (test) | | Metric | BLEU-1 | | Unit | SCALAR | | Direction | Higher is better | | Results loaded | 12 of 12 | | Coverage | All results currently returned for this benchmark | Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol. ## Metrics - [BLEU-1](): Higher is better; unit: SCALAR. - [BLEU-2](): Higher is better; unit: SCALAR. - [BLEU-3](): Higher is better; unit: SCALAR. - [BLEU-4](): Higher is better; unit: SCALAR. - [METEOR](): Higher is better; unit: SCALAR. - [ROUGE-L](): Higher is better; unit: SCALAR. - [CIDEr](): Higher is better; unit: SCALAR. - [Efficiency (%)](): Higher is better; unit: PERCENT. - [Human Score](): Higher is better; unit: SCALAR. ## Recorded results | Method | BLEU-1 | Result date | Report paper | | --- | --- | --- | --- | | EarthVLNet | 0.5726 | Not recorded | 695de86653fdd5c3617439f6 | | EarthVLNet w.o. seg | 0.5653 | Not recorded | 695de86653fdd5c3617439f6 | | LLaVANeXT | 0.5619 | Not recorded | 695de86653fdd5c3617439f6 | | GeoChat | 0.561 | Not recorded | 695de86653fdd5c3617439f6 | | ViP-LLaVA | 0.5601 | Not recorded | 695de86653fdd5c3617439f6 | | InstructBLIP | 0.5491 | Not recorded | 695de86653fdd5c3617439f6 | | LXMERT | 0.5393 | Not recorded | 695de86653fdd5c3617439f6 | | BUTD | 0.5124 | Not recorded | 695de86653fdd5c3617439f6 | | ALBEF | 0.5058 | Not recorded | 695de86653fdd5c3617439f6 | | BLIP-2 | 0.4777 | Not recorded | 695de86653fdd5c3617439f6 | | Claude3 Opus | 0.2564 | Not recorded | 695de86653fdd5c3617439f6 | | GPT-4o | 0.2111 | Not recorded | 695de86653fdd5c3617439f6 | ### EarthVLNet BLEU-1: 0.5726. Result ID: 6997e3fc5698665ec43cf854. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Seg":true,"Params":"6.9B"}. This record does not assert independent reproduction. ### EarthVLNet w.o. seg BLEU-1: 0.5653. Result ID: 6997e3fc5698665ec43cf853. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Seg":false,"Params":"6.9B"}. This record does not assert independent reproduction. ### LLaVANeXT BLEU-1: 0.5619. Result ID: 6997e3fc5698665ec43cf850. [Reporting paper]() Recorded evaluation configuration: {"Seg":false,"Params":"7.2B"}. This record does not assert independent reproduction. ### GeoChat BLEU-1: 0.561. Result ID: 6997e3fc5698665ec43cf84e. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Seg":false,"Params":"7.2B"}. This record does not assert independent reproduction. ### ViP-LLaVA BLEU-1: 0.5601. Result ID: 6997e3fc5698665ec43cf84f. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Seg":false,"Params":"7.2B"}. This record does not assert independent reproduction. ### InstructBLIP BLEU-1: 0.5491. Result ID: 6997e3fc5698665ec43cf84d. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Seg":false,"Params":"4.0B"}. This record does not assert independent reproduction. ### LXMERT BLEU-1: 0.5393. Result ID: 6997e3fc5698665ec43cf84a. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Seg":true,"Params":"114.3M"}. This record does not assert independent reproduction. ### BUTD BLEU-1: 0.5124. Result ID: 6997e3fc5698665ec43cf849. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Seg":true,"Params":"43.6M"}. This record does not assert independent reproduction. ### ALBEF BLEU-1: 0.5058. Result ID: 6997e3fc5698665ec43cf84b. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Seg":false,"Params":"290.6M"}. This record does not assert independent reproduction. ### BLIP-2 BLEU-1: 0.4777. Result ID: 6997e3fc5698665ec43cf84c. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Seg":false,"Params":"3.9B"}. This record does not assert independent reproduction. ### Claude3 Opus BLEU-1: 0.2564. Result ID: 6997e3fc5698665ec43cf852. [Reporting paper]() Recorded evaluation configuration: {"Seg":false,"Params":"-"}. This record does not assert independent reproduction. ### GPT-4o BLEU-1: 0.2111. Result ID: 6997e3fc5698665ec43cf851. [Reporting paper]() Recorded evaluation configuration: {"Seg":false,"Params":"-"}. This record does not assert independent reproduction. Page 1 of 1. This is an index page, not the full collection.