Open-ended Visual Question Answering on EarthVLSet 1.0 (test)
0.5726BLEU-1EarthVLNet
Evaluation results
| Method | Links | |||||||||
|---|---|---|---|---|---|---|---|---|---|---|
| EarthVLNetSeg=true, Params=6.9B2026.01 | 0.5726 | 0.4229 | 0.3211 | 0.2483 | 0.252 | 0.4025 | 0.3661 | 80.44 | 4.25 | |
| EarthVLNet w.o. segSeg=false, Params=6.9B2026.01 | 0.5653 | 0.414 | 0.3115 | 0.2417 | 0.2496 | 0.3976 | 0.3552 | 77.94 | 3.98 | |
| LLaVANeXTSeg=false, Params=7.2B2026.01 | 0.5619 | 0.4128 | 0.3106 | 0.2366 | 0.2493 | 0.3994 | 0.352 | 72.69 | 3.17 | |
| GeoChatSeg=false, Params=7.2B2026.01 | 0.561 | 0.4108 | 0.3118 | 0.2373 | 0.2489 | 0.3925 | 0.3504 | 73.61 | 3.44 | |
| ViP-LLaVASeg=false, Params=7.2B2026.01 | 0.5601 | 0.4094 | 0.301 | 0.2294 | 0.2316 | 0.3958 | 0.3492 | 72.8 | 3.64 | |
| InstructBLIPSeg=false, Params=4.0B2026.01 | 0.5491 | 0.3989 | 0.2965 | 0.2204 | 0.2276 | 0.3938 | 0.3392 | 70.69 | 3.35 | |
| LXMERTSeg=true, Params=114.3M2026.01 | 0.5393 | 0.3878 | 0.2869 | 0.2156 | 0.2429 | 0.3869 | 0.3031 | 76.42 | 3.73 | |
| BUTDSeg=true, Params=43.6M2026.01 | 0.5124 | 0.3667 | 0.2718 | 0.2062 | 0.2511 | 0.379 | 0.2788 | 76.74 | 3.66 | |
| ALBEFSeg=false, Params=290.6M2026.01 | 0.5058 | 0.3515 | 0.2495 | 0.1797 | 0.2365 | 0.3741 | 0.2564 | 67.21 | 3.11 | |
| BLIP-2Seg=false, Params=3.9B2026.01 | 0.4777 | 0.3298 | 0.2344 | 0.1684 | 0.1871 | 0.3419 | 0.2015 | 65.43 | 3.17 | |
| Claude3 OpusSeg=false, Params=-2026.01 | 0.2564 | 0.1532 | 0.0257 | 0.0229 | 0.127 | 0.2551 | 0.1844 | 59.22 | 2.94 | |
| GPT-4oSeg=false, Params=-2026.01 | 0.2111 | 0.1327 | 0.0184 | 0.0127 | 0.1392 | 0.1871 | 0.1239 | 56.74 | 2.81 |