Object Recognition on CC3M (test)
0.738RecallNXTP
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| NXTPmodels (vision + lang)=ViT L-14 + Langtruncated, data scale=3M, #params (B)=1.78, top-k=102023.12 | 0.738 | 0.53 | 0.611 | |
| NXTPmodels (vision + lang)=ViT L-14 + Langtruncated, data scale=70M, #params (B)=1.78, top-k=102023.12 | 0.722 | 0.512 | 0.593 | |
| CaSEDmodels (vision + lang)=ViT L-14 + Retrieval, data scale=403M, #params (B)=0.43, top-k=102023.12 | 0.653 | 0.481 | 0.548 | |
| CaSEDmodels (vision + lang)=ViT L-14 + Retrieval, data scale=12M, #params (B)=0.43, top-k=102023.12 | 0.648 | 0.471 | 0.54 | |
| InstructBLIPmodels (vision + lang)=ViT g-14 + Flant5xxl, prompt=caption, data scale=129M, #params (B)=12.3, top-k=102023.12 | 0.639 | 0.487 | 0.546 | |
| LLaVA 1.0models (vision + lang)=ViT L-14 + LLAMA 2, prompt=caption, data scale=753K, #params (B)=13.3, top-k=102023.12 | 0.634 | 0.46 | 0.528 | |
| LLaVA 1.5models (vision + lang)=ViT L-14 + Vicuna, prompt=caption, data scale=1.2M, #params (B)=13.4, top-k=102023.12 | 0.632 | 0.453 | 0.522 | |
| BLIP-2models (vision + lang)=ViT g-14 + Flant5xxl, prompt=caption, data scale=129M, #params (B)=12.2, top-k=102023.12 | 0.6 | 0.539 | 0.561 | |
| InstructBLIPmodels (vision + lang)=ViT g-14 + Flant5xxl, prompt=list, data scale=129M, #params (B)=12.3, top-k=102023.12 | 0.596 | 0.554 | 0.567 | |
| LLaVA 1.0models (vision + lang)=ViT L-14 + LLAMA 2, prompt=instruct, data scale=753K, #params (B)=13.3, top-k=102023.12 | 0.588 | 0.45 | 0.505 | |
| CLIPmodels (vision + lang)=ViT L-14 + CLIPlang, data scale=400M, #params (B)=0.43, top-k=102023.12 | 0.575 | 0.448 | 0.499 | |
| LLaVA 1.5models (vision + lang)=ViT L-14 + Vicuna, prompt=instruct, data scale=1.2M, #params (B)=13.4, top-k=102023.12 | 0.572 | 0.498 | 0.522 | |
| Flamingo_openmodels (vision + lang)=ViT L-14 + MPT, prompt=list, data scale=2.1B, #params (B)=8.13, top-k=102023.12 | 0.554 | 0.569 | 0.553 | |
| Flamingo_openmodels (vision + lang)=ViT L-14 + LLAMA 1, prompt=caption, data scale=2.1B, #params (B)=8.34, top-k=102023.12 | 0.548 | 0.521 | 0.527 | |
| Flamingo_openmodels (vision + lang)=ViT L-14 + LLAMA 1, prompt=list, data scale=2.1B, #params (B)=8.34, top-k=102023.12 | 0.547 | 0.54 | 0.536 | |
| BLIP-2models (vision + lang)=ViT g-14 + Flant5xxl, prompt=list, data scale=129M, #params (B)=12.2, top-k=102023.12 | 0.544 | 0.557 | 0.542 | |
| LLaVA 1.0models (vision + lang)=ViT L-14 + LLAMA 2, prompt=list, data scale=753K, #params (B)=13.3, top-k=102023.12 | 0.54 | 0.528 | 0.526 | |
| LLaVA 1.5models (vision + lang)=ViT L-14 + Vicuna, prompt=list, data scale=1.2M, #params (B)=13.4, top-k=102023.12 | 0.538 | 0.515 | 0.518 | |
| Flamingo_openmodels (vision + lang)=ViT L-14 + MPT, prompt=caption, data scale=2.1B, #params (B)=8.13, top-k=102023.12 | 0.534 | 0.533 | 0.527 | |
| InstructBLIPmodels (vision + lang)=ViT g-14 + Flant5xxl, prompt=instruct, data scale=129M, #params (B)=12.3, top-k=102023.12 | 0.529 | 0.604 | 0.555 | |
| CLIPmodels (vision + lang)=ViT L-14 + CLIPlang, data scale=400M, #params (B)=0.43, top-k=102023.12 | 0.451 | 0.383 | 0.409 |