Phrase grounding on Flickr30k Entities (test)
87.4Recall@1FIBER-B
Evaluation Results
| Method | Links | ||||
|---|---|---|---|---|---|
| FIBER-BImage Backbone=Swin-B, #Pretrain Images (fine-grained)=860k2022.06 | 87.4 | — | 96.4 | 97.6 | |
| FIBERZero-shot=false2023.06 | 87.4 | — | 96.4 | 97.6 | |
| GLIP-LData=FourODs, GoldG, Cap24M2021.12 | 87.1 | — | 96.9 | 98.1 | |
| GLIPImage Backbone=Swin-L, #Pretrain Images (fine-grained)=27M2022.06 | 87.1 | — | 96.9 | 98.1 | |
| GLIPZero-shot=false2023.06 | 87.1 | — | 96.9 | 98.1 | |
| FIBER-B (w/o C.G. VLP)Image Backbone=Swin-B, #Pretrain Images (fine-grained)=860k, Note=without coarse-grained VL pretraining2022.06 | 86.5 | — | 96.4 | 97.7 | |
| GLIPImage Backbone=Swin-B, #Pretrain Images (fine-grained)=860k2022.06 | 86.1 | — | 95.5 | 96.4 | |
| GLIP-TData=O365, GoldG, Cap4M2021.12 | 85.7 | — | 95.8 | 97.2 | |
| GLIP-TData=O365, GoldG2021.12 | 85.5 | — | 95.4 | 96.6 | |
| GLIP-TData=GoldG2021.12 | 84.4 | — | 95.3 | 97 | |
| MDETR-ENB5†*Evaluation Protocol=ANY-BOX-PROTOCOL, Backbone=EfficientNet-B5, Pre-training=COCO, VG, Flickr 30k2021.04 | 84.3 | — | 93.9 | 95.8 | |
| MDETR-ENB5Data=GoldG+2021.12 | 84.3 | — | 93.9 | 95.8 | |
| MDETRImage Backbone=EN-B5, #Pretrain Images (fine-grained)=200k2022.06 | 84.3 | — | 93.9 | 95.8 | |
| MDETRZero-shot=false2023.06 | 84.3 | — | 93.9 | 95.8 | |
| MDETR-ENB3†*Evaluation Protocol=ANY-BOX-PROTOCOL, Backbone=EfficientNet-B3, Pre-training=COCO, VG, Flickr 30k2021.04 | 84 | — | 93.8 | 95.6 | |
| MDETR-R101†*Evaluation Protocol=MERGED-BOXES-PROTOCOL, Backbone=ResNet-101, Pre-training=COCO, VG, Flickr 30k2021.04 | 83.8 | — | 92.7 | 94.4 | |
| MDETR-R101†*Evaluation Protocol=ANY-BOX-PROTOCOL, Backbone=ResNet-101, Pre-training=COCO, VG, Flickr 30k2021.04 | 83.4 | — | 93.5 | 95.3 | |
| MDETR-RN101Data=GoldG+2021.12 | 83.4 | — | 93.5 | 95.3 | |
| KOSMOS-2Zero-shot=true2023.06 | 78.7 | — | 80.1 | 80.1 | |
| DDPNEvaluation Protocol=MERGED-BOXES-PROTOCOL, Backbone=ResNet-1012021.04 | 73.5 | — | — | — | |
| SimNet-CCAEvaluation Protocol=MERGED-BOXES-PROTOCOL, Backbone=ResNet-1012021.04 | 71.9 | — | — | — | |
| VisualBert+Evaluation Protocol=ANY-BOX-PROTOCOL, Backbone=ResNet-1012021.04 | 71.3 | — | 85 | 86.5 | |
| Visual-BERTImage Backbone=ResNet-101, #Pretrain Images (fine-grained)=120k2022.06 | 71.3 | — | 85 | 86.5 | |
| VisualBertZero-shot=false2023.06 | 71.3 | — | 85 | 86.5 | |
| BANEvaluation Protocol=ANY-BOX-PROTOCOL, Backbone=ResNet-1012021.04 | 69.7 | — | 84.2 | 86.4 | |
| FAOGEvaluation Protocol=MERGED-BOXES-PROTOCOL, Backbone=ResNet-1012021.04 | 68.7 | — | — | — | |
| CITEEvaluation Protocol=MERGED-BOXES-PROTOCOL, Backbone=ResNet-1012021.04 | 61.9 | — | — | — | |
| GRILLZero-shot=true2023.06 | 18.9 | — | 53.4 | 70.3 | |
| BANVision Backbone=Bottom-Up Attention (Anderson et al., 2018)2019.09 | — | 0.6969 | — | — | |
| DDPNVision Backbone=Bottom-Up Attention (Anderson et al., 2018)2019.09 | — | 0.733 | — | — | |
| Ferret-13B2024.04 | — | 84.76 | — | — | |
| Ferret-7B2024.04 | — | 82.21 | — | — | |
| Ferret-v2-13B2024.04 | — | 86.25 | — | — | |
| Ferret-v2-7B2024.04 | — | 85.83 | — | — | |
| Griffon v22024.04 | — | 84.8 | — | — | |
| Hard-Label (GloVe)Vision Backbone=Bottom-Up Attention (Anderson et al., 2018), Word Embeddings=GloVe (Pennington et al., 2014)2019.09 | — | 0.7188 | — | — | |
| Hard-Label (HL)Vision Backbone=Bottom-Up Attention (Anderson et al., 2018), Word Embeddings=ELMo (Peters et al., 2018)2019.09 | — | 0.7221 | — | — | |
| Hard-Label Chain CRF (HL-CCRF)Vision Backbone=Bottom-Up Attention (Anderson et al., 2018), Word Embeddings=ELMo (Peters et al., 2018), Transition Context=M2019.09 | — | 0.7226 | — | — | |
| LLaVA-G2024.04 | — | 83.62 | — | — | |
| MDETRSpecifically fine-tuned in the second stage=true2024.04 | — | 83.8 | — | — | |
| Phrase-Region CCAVision Backbone=Fast R-CNN (Girshick, 2015)2019.09 | — | 0.5585 | — | — | |
| QRC NetVision Backbone=Fast R-CNN (Girshick, 2015)2019.09 | — | 0.6514 | — | — | |
| Shikra-13B2024.04 | — | 78.44 | — | — | |
| Shikra-7B2024.04 | — | 76.54 | — | — | |
| Soft-Label (SL)Vision Backbone=Bottom-Up Attention (Anderson et al., 2018), Word Embeddings=ELMo (Peters et al., 2018)2019.09 | — | 0.7429 | — | — | |
| Soft-Label Chain CRF (SL-CCRF)Vision Backbone=Bottom-Up Attention (Anderson et al., 2018), Word Embeddings=ELMo (Peters et al., 2018), Transition Context=M2019.09 | — | 0.7469 | — | — | |
| Structured MatchingVision Backbone=Fast R-CNN (Girshick, 2015)2019.09 | — | 0.4208 | — | — | |
| UniTAB2024.04 | — | 79.58 | — | — | |
| VistaLLM2024.04 | — | 84.8 | — | — |