Multiple-choice Visual Question Answering on EarthVLSet
81.06OAEarthVLNet (ours)
Evaluation results
| Method | Links | |||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| EarthVLNet (ours)Segmentation Guidance=true, Params=6.9B2026.01 | 81.06 | 89.24 | 88.01 | 81.16 | 74.83 | 66.2 | 58.51 | 42.03 | 64.9 | 2.34 | 0.908 | 4.341 | 7.141 | |
| LLaVA-OVSegmentation Guidance=false, Params=8.0B2026.01 | 80.42 | 89.06 | 87.53 | 80.12 | 73.57 | 67.09 | 49.88 | 43.24 | 63.14 | 2.54 | 0.967 | 4.672 | 8.524 | |
| SOBASegmentation Guidance=true, Params=19.9M2026.01 | 79.95 | 88.6 | 86.38 | 80.23 | 73.18 | 67.21 | 51.1 | 39.7 | 64.66 | 2.482 | 0.905 | 4.654 | 7.858 | |
| ViP-LLaVASegmentation Guidance=false, Params=7.2B2026.01 | 79.78 | 88.57 | 87.67 | 79.16 | 73.13 | 67.11 | 52.36 | 42.24 | 62.96 | 2.574 | 0.985 | 4.902 | 8.434 | |
| EarthVLNet w.o. segSegmentation Guidance=false, Params=6.9B2026.01 | 79.63 | 88.41 | 86.25 | 79.02 | 72.88 | 66.57 | 52.3 | 39.4 | 63.43 | 2.636 | 1.013 | 4.808 | 8.042 | |
| LLaVANeXTSegmentation Guidance=false, Params=7.2B2026.01 | 79.32 | 87.92 | 86.71 | 78.88 | 72.74 | 66.64 | 51.87 | 38.33 | 64.33 | 2.721 | 1.133 | 4.901 | 8.644 | |
| LXMERTSegmentation Guidance=true, Params=87.6M2026.01 | 79.27 | 88.32 | 86.07 | 79.98 | 72.57 | 67.02 | 50.33 | 37.59 | 63.97 | 2.594 | 1.088 | 4.894 | 7.981 | |
| BUTDSegmentation Guidance=true, Params=12.3M2026.01 | 79.26 | 87.15 | 86.64 | 78.35 | 72.38 | 66.53 | 48.28 | 36.43 | 63.21 | 2.568 | 0.993 | 4.826 | 8.224 | |
| MCANSegmentation Guidance=true, Params=17.7M2026.01 | 79.15 | 88.19 | 86.93 | 80.1 | 72.88 | 67.14 | 51.38 | 39.61 | 63.87 | 2.577 | 0.984 | 4.907 | 8.438 | |
| GeoChatSegmentation Guidance=false, Params=7.2B2026.01 | 79.13 | 88.4 | 86.41 | 78.92 | 72.67 | 66.14 | 42.59 | 38.14 | 63.77 | 2.766 | 1.253 | 5.038 | 8.858 | |
| BANSegmentation Guidance=true, Params=30.2M2026.01 | 78.97 | 88.55 | 86.68 | 78.45 | 72.04 | 66.34 | 50.38 | 33.36 | 63.49 | 2.864 | 1.291 | 4.953 | 8.906 | |
| InstructBLIPSegmentation Guidance=false, Params=4.0B2026.01 | 78.04 | 87.57 | 86.21 | 76.72 | 72 | 64.31 | 43.75 | 39.17 | 55.26 | 2.758 | 0.98 | 5.309 | 9.138 | |
| D-VQASegmentation Guidance=true, Params=17.6M2026.01 | 77.8 | 85.39 | 86.23 | 78.37 | 71.84 | 59.58 | 47.84 | 33.05 | 60.91 | 2.848 | 1.122 | 5.414 | 8.273 | |
| RSIVQASegmentation Guidance=false, Params=72.5M2026.01 | 77.79 | 84.52 | 86.12 | 76.71 | 71.22 | 63.69 | 45.24 | 39.52 | 61.34 | 3.381 | 1.302 | 6.144 | 10.29 | |
| SANSegmentation Guidance=false, Params=37.3M2026.01 | 77.24 | 82.07 | 86.29 | 75.13 | 71.46 | 62.81 | 49.11 | 30.05 | 60.54 | 3.326 | 1.479 | 5.948 | 10.506 | |
| ALBEFSegmentation Guidance=false, Params=290.6M2026.01 | 74.21 | 81.42 | 85.19 | 76.29 | 62.92 | 48.94 | 44.31 | 29.93 | 51.95 | 2.686 | 1.158 | 4.942 | 8.094 | |
| MACSegmentation Guidance=false, Params=49.9M2026.01 | 73.89 | 78.53 | 84.49 | 73.36 | 59.29 | 54.4 | 51.47 | 32.94 | 57.92 | 3.379 | 1.818 | 6.008 | 9.499 | |
| RSVQASegmentation Guidance=false, Params=34.9M2026.01 | 73.45 | 79.22 | 85.52 | 71.61 | 68.16 | 54.94 | 26.07 | 26.71 | 50.42 | 4.012 | 2.106 | 7.182 | 11.581 | |
| BLIP-2Segmentation Guidance=false, Params=3.9B2026.01 | 69.43 | 83.38 | 85.22 | 71.61 | 67.5 | 39.68 | 16.24 | 15.73 | 26.1 | 3.726 | 2.106 | 6.25 | 11.581 | |
| Claude3 OpusSegmentation Guidance=false2026.01 | 63.78 | 84.22 | 74.52 | 50.08 | 37.09 | 57.02 | 23.93 | 16.39 | 34.81 | 3.248 | 2.329 | 4.506 | 10.76 | |
| GPT-4oSegmentation Guidance=false2026.01 | 61.15 | 84.55 | 70.63 | 48.83 | 36.74 | 50.44 | 28.94 | 15.05 | 28.95 | 3.507 | 2.331 | 5.707 | 12.56 |