Object Recognition on COCO (val)
76.5RecallOurs
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| Ourstop-k=102023.12 | 76.5 | 75.6 | 75.8 | |
| NXTPmodels (vision + lang)=ViT L-14 + Langtruncated, data scale=70M, #params (B)=1.78, top-k=102023.12 | 76.5 | 75.7 | 75.8 | |
| nxtp2023.12 | 76.5 | 75.6 | 75.8 | |
| NXTPmodels (vision + lang)=ViT L-14 + Langtruncated, data scale=3M, #params (B)=1.78, top-k=102023.12 | 70 | 71.2 | 70.2 | |
| InstructBLIPmodels (vision + lang)=ViT g-14 + Flant5xxl, prompt=caption, data scale=129M, #params (B)=12.3, top-k=102023.12 | 69 | 66.2 | 67.3 | |
| LLaVA 1.0models (vision + lang)=ViT L-14 + LLAMA 2, prompt=caption, data scale=753K, #params (B)=13.3, top-k=102023.12 | 68.8 | 66.8 | 67.5 | |
| LLaVA 1.5models (vision + lang)=ViT L-14 + Vicuna, prompt=caption, data scale=1.2M, #params (B)=13.4, top-k=102023.12 | 67.9 | 64.9 | 66.1 | |
| LLaVA 1.0models (vision + lang)=ViT L-14 + LLAMA 2, prompt=instruct, data scale=753K, #params (B)=13.3, top-k=102023.12 | 63.8 | 63.1 | 63.2 | |
| LLaVA 1.5models (vision + lang)=ViT L-14 + Vicuna, prompt=instruct, data scale=1.2M, #params (B)=13.4, top-k=102023.12 | 63 | 71.6 | 65.9 | |
| GPT-4V Previewprompt=instruct, top-k=102023.12 | 62.5 | 60.1 | 61 | |
| GPT-4V Previewprompt=instruct2023.12 | 62.5 | 60.1 | 61 | |
| CaSEDmodels (vision + lang)=ViT L-14 + Retrieval, data scale=403M, #params (B)=0.43, top-k=102023.12 | 61.6 | 62.9 | 62 | |
| InstructBLIPprompt=list, top-k=102023.12 | 61.3 | 89.7 | 72.5 | |
| InstructBLIPmodels (vision + lang)=ViT g-14 + Flant5xxl, prompt=list, data scale=129M, #params (B)=12.3, top-k=102023.12 | 61.3 | 89.7 | 72.5 | |
| InstructBLIPprompt=list2023.12 | 61.3 | 89.7 | 72.5 | |
| BLIP-2models (vision + lang)=ViT g-14 + Flant5xxl, prompt=caption, data scale=129M, #params (B)=12.2, top-k=102023.12 | 60 | 89.3 | 71.4 | |
| LLaVA 1.5models (vision + lang)=ViT L-14 + Vicuna, prompt=list, data scale=1.2M, #params (B)=13.4, top-k=102023.12 | 59.1 | 78.3 | 66.5 | |
| CaSEDmodels (vision + lang)=ViT L-14 + Retrieval, data scale=12M, #params (B)=0.43, top-k=102023.12 | 58.2 | 59.2 | 58.4 | |
| LLaVA 1.0models (vision + lang)=ViT L-14 + LLAMA 2, prompt=list, data scale=753K, #params (B)=13.3, top-k=102023.12 | 58 | 80.3 | 66.6 | |
| InstructBLIPmodels (vision + lang)=ViT g-14 + Flant5xxl, prompt=instruct, data scale=129M, #params (B)=12.3, top-k=102023.12 | 56.9 | 87.9 | 68.6 | |
| Flamingo_open w/ MPTprompt=list, top-k=102023.12 | 55.6 | 79.4 | 64.7 | |
| Flamingo_openmodels (vision + lang)=ViT L-14 + MPT, prompt=list, data scale=2.1B, #params (B)=8.13, top-k=102023.12 | 55.6 | 79.3 | 64.6 | |
| Flamingo_open w/ MPTprompt=list2023.12 | 55.6 | 79.4 | 64.7 | |
| Flamingo_openmodels (vision + lang)=ViT L-14 + MPT, prompt=caption, data scale=2.1B, #params (B)=8.13, top-k=102023.12 | 55.4 | 75.4 | 63.3 | |
| Flamingo_openmodels (vision + lang)=ViT L-14 + LLAMA 1, prompt=caption, data scale=2.1B, #params (B)=8.34, top-k=102023.12 | 55.3 | 69.7 | 61.1 | |
| Flamingo_openmodels (vision + lang)=ViT L-14 + LLAMA 1, prompt=list, data scale=2.1B, #params (B)=8.34, top-k=102023.12 | 54.9 | 72.1 | 61.8 | |
| CLIPtop-k=102023.12 | 52.5 | 56.2 | 54 | |
| CLIPmodels (vision + lang)=ViT L-14 + CLIPlang, data scale=400M, #params (B)=0.43, top-k=102023.12 | 52.5 | 56.2 | 54 | |
| CLIP2023.12 | 52.5 | 56.2 | 54 | |
| BLIP-2models (vision + lang)=ViT g-14 + Flant5xxl, prompt=list, data scale=129M, #params (B)=12.2, top-k=102023.12 | 49.4 | 87.1 | 62.3 | |
| CLIPmodels (vision + lang)=ViT L-14 + CLIPlang, data scale=400M, #params (B)=0.43, top-k=102023.12 | 42.9 | 48.3 | 45 |