List your company
ResearchFormerly WizWand

Phrase grounding on Flickr30k Entities (test)

87.4Recall@1FIBER-B
16.1634.65553.1571.645Apr 26, 2021Sep 4, 2021Jan 14, 2022May 26, 2022Oct 5, 2022Feb 14, 2023Jun 26, 2023
Updated 8mo ago

Evaluation results

MethodLinks
FIBER-BImage Backbone=Swin-B, #Pretrain Images (fine-grained)=860k2022.0687.4—96.497.6
FIBERZero-shot=false2023.0687.4—96.497.6
GLIP-LData=FourODs, GoldG, Cap24M2021.1287.1—96.998.1
GLIPImage Backbone=Swin-L, #Pretrain Images (fine-grained)=27M2022.0687.1—96.998.1
GLIPZero-shot=false2023.0687.1—96.998.1
FIBER-B (w/o C.G. VLP)Image Backbone=Swin-B, #Pretrain Images (fine-grained)=860k, Note=without coarse-grained VL pretraining2022.0686.5—96.497.7
GLIPImage Backbone=Swin-B, #Pretrain Images (fine-grained)=860k2022.0686.1—95.596.4
GLIP-TData=O365, GoldG, Cap4M2021.1285.7—95.897.2
GLIP-TData=O365, GoldG2021.1285.5—95.496.6
GLIP-TData=GoldG2021.1284.4—95.397
MDETR-ENB5†*Evaluation Protocol=ANY-BOX-PROTOCOL, Backbone=EfficientNet-B5, Pre-training=COCO, VG, Flickr 30k2021.0484.3—93.995.8
MDETR-ENB5Data=GoldG+2021.1284.3—93.995.8
MDETRImage Backbone=EN-B5, #Pretrain Images (fine-grained)=200k2022.0684.3—93.995.8
MDETRZero-shot=false2023.0684.3—93.995.8
MDETR-ENB3†*Evaluation Protocol=ANY-BOX-PROTOCOL, Backbone=EfficientNet-B3, Pre-training=COCO, VG, Flickr 30k2021.0484—93.895.6
MDETR-R101†*Evaluation Protocol=MERGED-BOXES-PROTOCOL, Backbone=ResNet-101, Pre-training=COCO, VG, Flickr 30k2021.0483.8—92.794.4
MDETR-R101†*Evaluation Protocol=ANY-BOX-PROTOCOL, Backbone=ResNet-101, Pre-training=COCO, VG, Flickr 30k2021.0483.4—93.595.3
MDETR-RN101Data=GoldG+2021.1283.4—93.595.3
KOSMOS-2Zero-shot=true2023.0678.7—80.180.1
DDPNEvaluation Protocol=MERGED-BOXES-PROTOCOL, Backbone=ResNet-1012021.0473.5———
SimNet-CCAEvaluation Protocol=MERGED-BOXES-PROTOCOL, Backbone=ResNet-1012021.0471.9———
VisualBert+Evaluation Protocol=ANY-BOX-PROTOCOL, Backbone=ResNet-1012021.0471.3—8586.5
Visual-BERTImage Backbone=ResNet-101, #Pretrain Images (fine-grained)=120k2022.0671.3—8586.5
VisualBertZero-shot=false2023.0671.3—8586.5
BANEvaluation Protocol=ANY-BOX-PROTOCOL, Backbone=ResNet-1012021.0469.7—84.286.4
FAOGEvaluation Protocol=MERGED-BOXES-PROTOCOL, Backbone=ResNet-1012021.0468.7———
CITEEvaluation Protocol=MERGED-BOXES-PROTOCOL, Backbone=ResNet-1012021.0461.9———
GRILLZero-shot=true2023.0618.9—53.470.3
BANVision Backbone=Bottom-Up Attention (Anderson et al., 2018)2019.09—0.6969——
DDPNVision Backbone=Bottom-Up Attention (Anderson et al., 2018)2019.09—0.733——
Ferret-13B2024.04—84.76——
Ferret-7B2024.04—82.21——
Ferret-v2-13B2024.04—86.25——
Ferret-v2-7B2024.04—85.83——
Griffon v22024.04—84.8——
Hard-Label (GloVe)Vision Backbone=Bottom-Up Attention (Anderson et al., 2018), Word Embeddings=GloVe (Pennington et al., 2014)2019.09—0.7188——
Hard-Label (HL)Vision Backbone=Bottom-Up Attention (Anderson et al., 2018), Word Embeddings=ELMo (Peters et al., 2018)2019.09—0.7221——
Hard-Label Chain CRF (HL-CCRF)Vision Backbone=Bottom-Up Attention (Anderson et al., 2018), Word Embeddings=ELMo (Peters et al., 2018), Transition Context=M2019.09—0.7226——
LLaVA-G2024.04—83.62——
MDETRSpecifically fine-tuned in the second stage=true2024.04—83.8——
Phrase-Region CCAVision Backbone=Fast R-CNN (Girshick, 2015)2019.09—0.5585——
QRC NetVision Backbone=Fast R-CNN (Girshick, 2015)2019.09—0.6514——
Shikra-13B2024.04—78.44——
Shikra-7B2024.04—76.54——
Soft-Label (SL)Vision Backbone=Bottom-Up Attention (Anderson et al., 2018), Word Embeddings=ELMo (Peters et al., 2018)2019.09—0.7429——
Soft-Label Chain CRF (SL-CCRF)Vision Backbone=Bottom-Up Attention (Anderson et al., 2018), Word Embeddings=ELMo (Peters et al., 2018), Transition Context=M2019.09—0.7469——
Structured MatchingVision Backbone=Fast R-CNN (Girshick, 2015)2019.09—0.4208——
UniTAB2024.04—79.58——
VistaLLM2024.04—84.8——