# SigLIP 2: Multilingual Vision-Language Encoders with Improved Semantic Understanding, Localization, and Dense Features

Source: [SigLIP 2: Multilingual Vision-Language Encoders with Improved Semantic Understanding, Localization, and Dense Features](<https://www.sota2.com/research/paper/68fa770a1e209131cea70ad4>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2025-10-26T04:08:58.245Z

Authors: Michael Tschannen, Alexey Gritsenko, Xiao Wang, Muhammad Ferjad Naeem, Ibrahim Alabdulmohsin, Nikhil Parthasarathy, Talfan Evans, Lucas Beyer, Ye Xia, Basil Mustafa, Olivier H\\'enaff, Jeremiah Harmsen, Andreas Steiner, Xiaohua Zhai

Published: 2025-02-20T18:08:29.000Z

## Abstract

We introduce SigLIP 2, a family of new multilingual vision-language encoders that build on the success of the original SigLIP. In this second iteration, we extend the original image-text training objective with several prior, independently developed techniques into a unified recipe -- this includes captioning-based pretraining, self-supervised losses (self-distillation, masked prediction) and online data curation. With these changes, SigLIP 2 models outperform their SigLIP counterparts at all model scales in core capabilities, including zero-shot classification, image-text retrieval, and transfer performance when extracting visual representations for Vision-Language Models (VLMs). Furthermore, the new training recipe leads to significant improvements on localization and dense prediction tasks. We also train variants which support multiple resolutions and preserve the input's native aspect ratio. Finally, we train on a more diverse data-mixture that includes de-biasing techniques, leading to much better multilingual understanding and improved fairness. To allow users to trade off inference cost with performance, we release model checkpoints at four sizes: ViT-B (86M), L (303M), So400m (400M), and g (1B).

## Recorded benchmark results

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

- [Image Classification on SUN397 (Accuracy)](<https://www.sota2.com/research/sota/image-classification-on-sun397-accuracy.md>): Image Classification; SUN397; Accuracy: 97.8.

- [Image Classification on ImageNet-R (Accuracy)](<https://www.sota2.com/research/sota/image-classification-on-imagenet-r-accuracy.md>): Image Classification; ImageNet-R; Accuracy: 96.6.

- [Video retrieval on UCF101](<https://www.sota2.com/research/sota/video-retrieval-on-ucf101.md>): Video Retrieval; UCF101; Top-1 Acc: 90.7.

- [Fine-grained Classification on Food101](<https://www.sota2.com/research/sota/fine-grained-classification-on-food101.md>): Fine grained classification; Food101; Top-1 Acc: 94.7.

- [Object Classification on ImageNet-1K](<https://www.sota2.com/research/sota/object-classification-on-imagenet-1k.md>): Object Classification; ImageNet-1K; Top-1 Acc: 88.

- [OCR Benchmarking on OCRBench v2 (en/cn metrics)](<https://www.sota2.com/research/sota/ocr-benchmarking-on-ocrbench-v2-en-cn-metrics.md>): OCR Benchmarking; OCRBench v2; Accuracy: 62.6.

- [Image Classification on Onion (test)](<https://www.sota2.com/research/sota/image-classification-on-onion-test.md>): Image Classification; Onion (test); Accuracy: 64.71.

- [Image Classification on Wheat (test)](<https://www.sota2.com/research/sota/image-classification-on-wheat-test.md>): Image Classification; Wheat (test); Accuracy: 90.1.

- [Image Classification on ImageNet 1K (val) (Zero-Shot Accuracy)](<https://www.sota2.com/research/sota/image-classification-on-imagenet-1k-val-zero-shot-accuracy.md>): Image Classification; ImageNet-1k (val); Zero-Shot Accuracy: 83.4.

- [Image-to-Text Retrieval on MS-COCO 2017 (val)](<https://www.sota2.com/research/sota/image-to-text-retrieval-on-ms-coco-2017-val.md>): Image-to-Text Retrieval; MS-COCO 2017 (val); R@1 (I2T): 22.

- [General Visual Question Answering on MME-P](<https://www.sota2.com/research/sota/general-visual-question-answering-on-mme-p.md>): General Visual Question Answering; MME-P; Rescaled Score: 1284.

- [Image Classification on SUN397 (Top-1 Accuracy)](<https://www.sota2.com/research/sota/image-classification-on-sun397-top-1-accuracy.md>): Image Classification; SUN397; Top-1 Accuracy: 78.92.

- [3D Awareness Probing (Texture) on Feat2GS average of 6 datasets](<https://www.sota2.com/research/sota/3d-awareness-probing-texture-on-feat2gs-average-of-6-datasets.md>): 3D Awareness Probing (Texture); Feat2GS average of 6 datasets; PSNR: 18.11.

- [Image Classification on Histology](<https://www.sota2.com/research/sota/image-classification-on-histology.md>): Image Classification; Histology; Accuracy: 9.

- [Image Classification on ImageNet-1K (val) (Accuracy Only)](<https://www.sota2.com/research/sota/image-classification-on-imagenet-1k-val-accuracy-only.md>): Image Classification; ImageNet-1k (val); Top-1 Accuracy: 88.9.

- [UI Screen Classification on RICO SCA](<https://www.sota2.com/research/sota/ui-screen-classification-on-rico-sca.md>): UI Screen Classification; RICO SCA; Accuracy: 93.3.

- [Vision-Language Perception on MME Perception](<https://www.sota2.com/research/sota/vision-language-perception-on-mme-perception.md>): Vision-Language Perception; MME Perception; Perception Score: 1389.5.

- [Zero-shot Image-Text Retrieval on MSCOCO](<https://www.sota2.com/research/sota/zero-shot-image-text-retrieval-on-mscoco.md>): Zero-shot Image-Text Retrieval; MSCOCO; Accuracy: 82.8.

- [SAE Interpretability and Faithfulness Evaluation on SigLIP2-B 16 activations](<https://www.sota2.com/research/sota/sae-interpretability-and-faithfulness-evaluation-on-siglip2-b-16-activations.md>): SAE Interpretability and Faithfulness Evaluation; SigLIP2-B 16 activations; LLM Rank: 6.

- [Classification on ImageNet-1k (val) (Top-1 Accuracy)](<https://www.sota2.com/research/sota/classification-on-imagenet-1k-val-top-1-accuracy.md>): Classification; ImageNet-1k (val); Top-1 Accuracy: 85.

- [Image-Text Alignment Scoring on Aircraft synthetic images 70 images (test)](<https://www.sota2.com/research/sota/image-text-alignment-scoring-on-aircraft-synthetic-images-70-images-test.md>): Image-Text Alignment Scoring; Aircraft synthetic images 70 images (test); Mean Alignment Score: 99.9.

- [Image-Text Alignment Scoring on Car synthetic 70 images (test)](<https://www.sota2.com/research/sota/image-text-alignment-scoring-on-car-synthetic-70-images-test.md>): Image-Text Alignment Scoring; Car synthetic 70 images (test); Mean Score: 99.

- [Image-Text Alignment Scoring on Animal synthetic images 70 images (test)](<https://www.sota2.com/research/sota/image-text-alignment-scoring-on-animal-synthetic-images-70-images-test.md>): Image-Text Alignment Scoring; Animal synthetic images 70 images (test); Mean Score: 99.9.

- [Image-Text Scoring Variability Analysis on Aircraft 70 synthetic images (test)](<https://www.sota2.com/research/sota/image-text-scoring-variability-analysis-on-aircraft-70-synthetic-images-test.md>): Image-Text Scoring Variability Analysis; Aircraft 70 synthetic images (test); Range: 1.

- [Image-Text Scoring Variability Analysis on Car image dataset 70 synthetic (test)](<https://www.sota2.com/research/sota/image-text-scoring-variability-analysis-on-car-image-dataset-70-synthetic-test.md>): Image-Text Scoring Variability Analysis; Car image dataset 70 synthetic (test); Range: 11.

- [Image-Text Scoring Variability Analysis on Animal image dataset 70 synthetic images (test)](<https://www.sota2.com/research/sota/image-text-scoring-variability-analysis-on-animal-image-dataset-70-synthetic-images-test.md>): Image-Text Scoring Variability Analysis; Animal image dataset 70 synthetic images (test); Range: 3.7.

- [Image Classification on CIFAR-10 EN](<https://www.sota2.com/research/sota/image-classification-on-cifar-10-en.md>): Image Classification; CIFAR-10 EN; Accuracy: 93.91.

- [Image Classification on CIFAR-10 FR](<https://www.sota2.com/research/sota/image-classification-on-cifar-10-fr.md>): Image Classification; CIFAR-10 FR; Accuracy: 92.21.

- [Image Classification on CIFAR-10 ZH](<https://www.sota2.com/research/sota/image-classification-on-cifar-10-zh.md>): Image Classification; CIFAR-10 ZH; Accuracy: 93.85.

- [Tactile-Visual Embedding Alignment on GelSight Mini](<https://www.sota2.com/research/sota/tactile-visual-embedding-alignment-on-gelsight-mini.md>): Tactile-Visual Embedding Alignment; GelSight Mini; Accuracy: 27.8.

- [Image Classification on ObjectNet](<https://www.sota2.com/research/sota/image-classification-on-objectnet.md>): Image Classification; ObjectNet; Accuracy: 91.5.

- [Image Classification on ImageNet1k](<https://www.sota2.com/research/sota/image-classification-on-imagenet1k.md>): Classification; ImageNet1K; Accuracy: 88.

- [KNN classification on ImageNet-1k (val)](<https://www.sota2.com/research/sota/knn-classification-on-imagenet-1k-val.md>): KNN Classification; ImageNet-1k (val); Top-1 Accuracy: 83.2.

- [Open-vocabulary Semantic Segmentation on Pascal VOC 21](<https://www.sota2.com/research/sota/open-vocabulary-semantic-segmentation-on-pascal-voc-21.md>): Open Vocabulary Semantic Segmentation; Pascal VOC 21; mIoU: 82.3.

- [Video Action Classification on COIN](<https://www.sota2.com/research/sota/video-action-classification-on-coin.md>): Video Action Classification; COIN; Top-1 Acc: 95.1.

- [Classification on FGVC Aircraft](<https://www.sota2.com/research/sota/classification-on-fgvc-aircraft.md>): Classification; FGVC Aircraft; Top-1 Accuracy: 54.8.

- [Image Classification on ImageNet 1K (test) (Top-1 Accuracy and GAIN)](<https://www.sota2.com/research/sota/image-classification-on-imagenet-1k-test-top-1-accuracy-and-gain.md>): Image Classification; ImageNet 1k (test); Top-1 Acc: 87.

- [Classification on SUN 397 (Top-1 Accuracy)](<https://www.sota2.com/research/sota/classification-on-sun-397-top-1-accuracy.md>): Classification; SUN 397; Top-1 Accuracy: 72.7.

- [Image Classification on Flowers VTAB](<https://www.sota2.com/research/sota/image-classification-on-flowers-vtab.md>): Image Classification; Flowers VTAB; Accuracy: 95.9.

- [Image-Text Retrieval on MSCOCO (R@1, R@5, nDCG@10)](<https://www.sota2.com/research/sota/image-text-retrieval-on-mscoco-r-at-1-r-at-5-ndcg-at-10.md>): Image-Text Retrieval; MSCOCO; Recall@1: 52.1.

- [Image Classification on MedMNIST 2D OrganAMNIST](<https://www.sota2.com/research/sota/image-classification-on-medmnist-2d-organamnist.md>): Image Classification; MedMNIST 2D OrganAMNIST; Accuracy: 95.6.

- [Image Classification on ImageNet 256x256 (val)](<https://www.sota2.com/research/sota/image-classification-on-imagenet-256x256-val.md>): Image Classification; ImageNet 256x256 (val); Top-1 Acc: 83.4.

- [Image Classification on Chili (test)](<https://www.sota2.com/research/sota/image-classification-on-chili-test.md>): Image Classification; Chili (test); Accuracy: 75.24.

- [Web UI Localization on Screenspot V2](<https://www.sota2.com/research/sota/web-ui-localization-on-screenspot-v2.md>): Web UI Localization; ScreenSpot v2; Center Accuracy: 81.4.

- [Web UI Question Answering on VisualWebBench](<https://www.sota2.com/research/sota/web-ui-question-answering-on-visualwebbench.md>): Web UI Question Answering; VisualWebBench; Accuracy: 65.4.

- [Web UI Question Answering on WebSRC](<https://www.sota2.com/research/sota/web-ui-question-answering-on-websrc.md>): Web UI Question Answering; WebSRC; Accuracy: 0.883.

- [Image Classification on MedMNIST 2D OCTMNIST](<https://www.sota2.com/research/sota/image-classification-on-medmnist-2d-octmnist.md>): Image Classification; MedMNIST 2D OCTMNIST; Accuracy: 88.3.

- [Image Classification on MedMNIST BloodMNIST 2D](<https://www.sota2.com/research/sota/image-classification-on-medmnist-bloodmnist-2d.md>): Image Classification; MedMNIST BloodMNIST 2D; Accuracy: 98.4.

- [Image Classification on MedMNIST 2D OrganCMNIST](<https://www.sota2.com/research/sota/image-classification-on-medmnist-2d-organcmnist.md>): Image Classification; MedMNIST 2D OrganCMNIST; Accuracy: 94.3.

- [Text-to-Image Retrieval on MS-COCO 2017 (val)](<https://www.sota2.com/research/sota/text-to-image-retrieval-on-ms-coco-2017-val.md>): Text-to-Image Retrieval; MS-COCO 2017 (val); R@1 (T2I): 21.5.

- [Image Captioning on Captioning Suite](<https://www.sota2.com/research/sota/image-captioning-on-captioning-suite.md>): Image Captioning; Captioning Suite; NoCaps Score: 84.1.

- [Overall Performance (Average) on Real-world Dexterous Manipulation Dexmate Vega](<https://www.sota2.com/research/sota/overall-performance-average-on-real-world-dexterous-manipulation-dexmate-vega.md>): Overall Performance (Average); Real-world Dexterous Manipulation Dexmate Vega; Average Success Rate: 43.4.

- [Turn On Lamp on Real-world Dexterous Manipulation Dexmate Vega](<https://www.sota2.com/research/sota/turn-on-lamp-on-real-world-dexterous-manipulation-dexmate-vega.md>): Turn On Lamp; Real-world Dexterous Manipulation Dexmate Vega; Success Rate: 50.

- [Semantic Correspondence on SPair](<https://www.sota2.com/research/sota/semantic-correspondence-on-spair.md>): Semantic Correspondence; SPair; Avg. Recall: 29.9.

- [Document Layout Analysis on DocBank](<https://www.sota2.com/research/sota/document-layout-analysis-on-docbank.md>): Document Layout Analysis; DocBank; mAP: 51.7.

- [Text-to-Image Retrieval on Retrieval VQA Visual Genome (test)](<https://www.sota2.com/research/sota/text-to-image-retrieval-on-retrieval-vqa-visual-genome-test.md>): Text-to-Image Retrieval; Retrieval VQA Visual Genome (test); Count Accuracy: 66.6.

- [Zero-shot Image-Text Retrieval on IIW](<https://www.sota2.com/research/sota/zero-shot-image-text-retrieval-on-iiw.md>): Zero-shot Image-Text Retrieval; IIW; Zero-shot Accuracy (IIW): 78.1.

- [Athlete Detection on Soccer Pretraining Dataset](<https://www.sota2.com/research/sota/athlete-detection-on-soccer-pretraining-dataset.md>): Athlete Detection; Soccer Pretraining Dataset; AP@50: 72.3.

- [Depth Estimation on OctoSense degraded (test)](<https://www.sota2.com/research/sota/depth-estimation-on-octosense-degraded-test.md>): Depth Estimation; OctoSense degraded (test); Depth (m): 7.55.

- [Lines Detection on Soccer Pretraining Dataset](<https://www.sota2.com/research/sota/lines-detection-on-soccer-pretraining-dataset.md>): Lines Detection; Soccer Pretraining Dataset; Accuracy: 93.6.

- [Viewpoint Grounding on ScanRefer](<https://www.sota2.com/research/sota/viewpoint-grounding-on-scanrefer.md>): Viewpoint Grounding; ScanRefer; R@1: 22.2.

- [Viewpoint Grounding on Nr3D](<https://www.sota2.com/research/sota/viewpoint-grounding-on-nr3d.md>): Viewpoint Grounding; Nr3D; Recall@1: 18.8.

- [Image Captioning on COCO (test) (CIDEr, BLEU@4)](<https://www.sota2.com/research/sota/image-captioning-on-coco-test-cider-bleu-at-4.md>): Captioning; COCO (test); CIDEr: 114.2.

- [Evidence Awareness on RCLIP-Bench Evidence Awareness V2](<https://www.sota2.com/research/sota/evidence-awareness-on-rclip-bench-evidence-awareness-v2.md>): Evidence Awareness; RCLIP-Bench Evidence Awareness V2; Score S: 92.7.

- [Image Classification on MMVP (Avg.)](<https://www.sota2.com/research/sota/image-classification-on-mmvp-avg.md>): Image Classification; MMVP; Average Score: 34.8.

- [Visual Grounding on RCLIP-Bench Visual Grounding V1](<https://www.sota2.com/research/sota/visual-grounding-on-rclip-bench-visual-grounding-v1.md>): Visual Grounding; RCLIP-Bench Visual Grounding V1; Success Rate (Att.): 92.7.

- [Visual Reasoning on RCLIP-Bench Visual Reasoning V3](<https://www.sota2.com/research/sota/visual-reasoning-on-rclip-bench-visual-reasoning-v3.md>): Visual Reasoning; RCLIP-Bench Visual Reasoning V3; Score (S): 89.1.

- [Composition-based Retrieval on MS-COCO](<https://www.sota2.com/research/sota/composition-based-retrieval-on-ms-coco.md>): Composition-based Retrieval; MS-COCO; Comp@1: 50.51.

- [Demand Prediction on H&M Personalized Fashion Recommendations (10 random train-test splits)](<https://www.sota2.com/research/sota/demand-prediction-on-h-and-m-personalized-fashion-recommendations-10-random-train-test-splits.md>): Demand Prediction; H&M Personalized Fashion Recommendations (10 random train-test splits); CRPS: 0.99.

- [Image Classification on Leaf Disease OOD (test)](<https://www.sota2.com/research/sota/image-classification-on-leaf-disease-ood-test.md>): Image Classification; Leaf Disease OOD (test); Accuracy W: 31.07.

- [Image-to-Text Retrieval on Flickr30k (Recall)](<https://www.sota2.com/research/sota/image-to-text-retrieval-on-flickr30k-recall.md>): Image-to-Text Retrieval; Flickr30K; Recall: 95.4.

- [Image-to-Text Retrieval on Flickr30k (R@1, R@5, R@25, R@50)](<https://www.sota2.com/research/sota/image-to-text-retrieval-on-flickr30k-r-at-1-r-at-5-r-at-25-r-at-50.md>): Image-to-Text Retrieval; Flickr30K; Recall@1: 90.1.

- [Text-to-Image Retrieval on NWPU](<https://www.sota2.com/research/sota/text-to-image-retrieval-on-nwpu.md>): Text-to-Image Retrieval; NWPU; Avg R@1/5/10: 36.7.

- [Text-to-Image Retrieval on Flickr30k (R@1, R@5, R@25, R@50)](<https://www.sota2.com/research/sota/text-to-image-retrieval-on-flickr30k-r-at-1-r-at-5-r-at-25-r-at-50.md>): Text-to-Image Retrieval; Flickr30K; R@1: 74.24.

- [Image Classification on CIFAR-10 (SW)](<https://www.sota2.com/research/sota/image-classification-on-cifar-10-sw.md>): Image Classification; CIFAR-10 (SW); Accuracy: 10.21.

- [Semantic Segmentation on GOOSE (val)](<https://www.sota2.com/research/sota/semantic-segmentation-on-goose-val.md>): Semantic segmentation; GOOSE (val); Composite mIoU: 72.9.

- [Video Retrieval on VATEX](<https://www.sota2.com/research/sota/video-retrieval-on-vatex.md>): Video Retrieval; VATEX; Recall@1: 27.52.

- [Document Understanding on DTCBench](<https://www.sota2.com/research/sota/document-understanding-on-dtcbench.md>): Document Understanding; DTCBench; Accuracy: 29.2.

- [Multilingual Text-to-Image Retrieval on RefCOCO](<https://www.sota2.com/research/sota/multilingual-text-to-image-retrieval-on-refcoco.md>): Multilingual Text-to-Image Retrieval; RefCOCO; Accuracy (EN): 61.37.

- [Multilingual Text-to-Image Retrieval on RefCOCO+](<https://www.sota2.com/research/sota/multilingual-text-to-image-retrieval-on-refcoco-plus.md>): Multilingual Text-to-Image Retrieval; RefCOCO+; Recall (EN): 30.96.

- [Multilingual Text-to-Image Retrieval on RefCOCOg](<https://www.sota2.com/research/sota/multilingual-text-to-image-retrieval-on-refcocog.md>): Multilingual Text-to-Image Retrieval; RefCOCOg; Retrieval Accuracy (EN): 56.

- [Tactile-Visual Embedding Alignment on 9DTact](<https://www.sota2.com/research/sota/tactile-visual-embedding-alignment-on-9dtact.md>): Tactile-Visual Embedding Alignment; 9DTact; Accuracy: 27.8.

- [Tactile-Visual Embedding Alignment on TransTac](<https://www.sota2.com/research/sota/tactile-visual-embedding-alignment-on-transtac.md>): Tactile-Visual Embedding Alignment; TransTac; Accuracy: 83.3.

- [Visual Question Answering on CMMMU](<https://www.sota2.com/research/sota/visual-question-answering-on-cmmmu.md>): Visual Question Answering; CMMMU; Accuracy: 21.4.

- [Image Classification on Oxford-IIIT Pet (Top-1 accuracy)](<https://www.sota2.com/research/sota/image-classification-on-oxford-iiit-pet-top-1-accuracy.md>): Image Classification; Oxford-IIIT Pet; Top-1 Accuracy: 95.4.

- [Image Classification on DermaMNIST](<https://www.sota2.com/research/sota/image-classification-on-dermamnist.md>): Image Classification; DermaMNIST; Accuracy: 81.8.

- [Image Retrieval on COCO (R@K and MR)](<https://www.sota2.com/research/sota/image-retrieval-on-coco-r-at-k-and-mr.md>): Image Retrieval; COCO; R@1: 56.55.

- [Multi-modal Understanding on MMMU (Accuracy)](<https://www.sota2.com/research/sota/multi-modal-understanding-on-mmmu-accuracy.md>): Multi-modal Understanding; MMMU; Accuracy: 48.3.

- [Compositional Reasoning on SugarCrepe++](<https://www.sota2.com/research/sota/compositional-reasoning-on-sugarcrepe-plus-plus.md>): Compositional Reasoning; SugarCrepe++; Average Performance: 74.4.

- [Image Classification on Resisc45 (vtab)](<https://www.sota2.com/research/sota/image-classification-on-resisc45-vtab.md>): Image Classification; resisc45 vtab; Accuracy: 75.9.

- [Multimodal Large Language Model Evaluation on MLLM Evaluation Suite](<https://www.sota2.com/research/sota/multimodal-large-language-model-evaluation-on-mllm-evaluation-suite.md>): Multimodal Large Language Model Evaluation; MLLM Evaluation Suite; Average Score (All): 55.4.

- [Image-Text Retrieval on Flickr30K (R@1, R@5, nDCG@10)](<https://www.sota2.com/research/sota/image-text-retrieval-on-flickr30k-r-at-1-r-at-5-ndcg-at-10.md>): Image-Text Retrieval; Flickr30K; Recall@1: 79.8.

- [Image Classification on Soursop (test)](<https://www.sota2.com/research/sota/image-classification-on-soursop-test.md>): Image Classification; Soursop (test); Accuracy: 97.68.

- [Open-vocabulary semantic segmentation on ADE20K 150](<https://www.sota2.com/research/sota/open-vocabulary-semantic-segmentation-on-ade20k-150.md>): Open Vocabulary Semantic Segmentation; ADE20K 150; mIoU: 38.8.

- [Classification on CBIS-DDSM (5-fold cross-validation)](<https://www.sota2.com/research/sota/classification-on-cbis-ddsm-5-fold-cross-validation.md>): Classification; CBIS-DDSM (5-fold cross-validation); AUROC: 82.3.

- [Image Classification on Describable Textures Dataset](<https://www.sota2.com/research/sota/image-classification-on-describable-textures-dataset.md>): Image Classification; Describable Textures Dataset; Accuracy: 70.2.

- [Image Classification on MedMNIST PneumoniaMNIST 2D](<https://www.sota2.com/research/sota/image-classification-on-medmnist-pneumoniamnist-2d.md>): Image Classification; MedMNIST PneumoniaMNIST 2D; Accuracy: 90.1.

- [Image-Text Retrieval on DOCCI-FG sentence-level](<https://www.sota2.com/research/sota/image-text-retrieval-on-docci-fg-sentence-level.md>): Image-Text Retrieval; DOCCI-FG sentence-level; T2I Recall@1: 23.2.

- [Classification on CUB 200 2011 (holdout set)](<https://www.sota2.com/research/sota/classification-on-cub-200-2011-holdout-set.md>): Classification; CUB 200 2011 (holdout set); Accuracy: 78.6.

- [Image Classification on MedMNIST 2D RetinaMNIST](<https://www.sota2.com/research/sota/image-classification-on-medmnist-2d-retinamnist.md>): Image Classification; MedMNIST 2D RetinaMNIST; Accuracy: 55.6.

- [Open-vocabulary semantic segmentation on Pascal Context 459](<https://www.sota2.com/research/sota/open-vocabulary-semantic-segmentation-on-pascal-context-459.md>): Open Vocabulary Semantic Segmentation; Pascal Context 459; mIoU: 24.1.

- [Text-to-image Retrieval on UCM caption](<https://www.sota2.com/research/sota/text-to-image-retrieval-on-ucm-caption.md>): Text-to-Image Retrieval; UCM caption; R@1/5/10: 68.52.

- [Document and OCR on TextVQA](<https://www.sota2.com/research/sota/document-and-ocr-on-textvqa.md>): Document and OCR; TextVQA; Accuracy Score: 54.7.

- [Image Classification on MedMNIST TissueMNIST 2D](<https://www.sota2.com/research/sota/image-classification-on-medmnist-tissuemnist-2d.md>): Image Classification; MedMNIST TissueMNIST 2D; Accuracy: 70.2.

- [Dispense Soap on Real-world Dexterous Manipulation Dexmate Vega](<https://www.sota2.com/research/sota/dispense-soap-on-real-world-dexterous-manipulation-dexmate-vega.md>): Dispense Soap; Real-world Dexterous Manipulation Dexmate Vega; Success Rate: 93.75.

- [Multi-modal Retrieval-Augmented Generation on MRAG-Bench](<https://www.sota2.com/research/sota/multi-modal-retrieval-augmented-generation-on-mrag-bench.md>): Multi-modal Retrieval-Augmented Generation; MRAG-Bench; Accuracy: 63.34.

- [Multimodal Understanding on Multimodal Evaluation Suite Table 4](<https://www.sota2.com/research/sota/multimodal-understanding-on-multimodal-evaluation-suite-table-4.md>): Multimodal Understanding; Multimodal Evaluation Suite Table 4; ALL AVG Score: 69.4.

- [Snippet localization on RefCT internal (test)](<https://www.sota2.com/research/sota/snippet-localization-on-refct-internal-test.md>): Snippet localization; RefCT internal (test); MAE (mm): 67.

- [Bounding box classification on BoxClass-CN566](<https://www.sota2.com/research/sota/bounding-box-classification-on-boxclass-cn566.md>): Bounding-box classification; BoxClass-CN566; Top-1 Accuracy: 63.6.

- [Classification on DermaMNIST 10% ratio Linear Evaluation](<https://www.sota2.com/research/sota/classification-on-dermamnist-10-percent-ratio-linear-evaluation.md>): Classification; DermaMNIST 10% ratio Linear Evaluation; Accuracy: 77.1.

- [Classification on RetinaMNIST Linear Evaluation 1% ratio](<https://www.sota2.com/research/sota/classification-on-retinamnist-linear-evaluation-1-percent-ratio.md>): Classification; RetinaMNIST Linear Evaluation 1% ratio; Accuracy: 43.8.

- [Image-to-Text Cross-lingual Retrieval on OOD handwriting set](<https://www.sota2.com/research/sota/image-to-text-cross-lingual-retrieval-on-ood-handwriting-set.md>): Image-to-Text Cross-lingual Retrieval; OOD handwriting set; Accuracy@1 (en-\>zh): 36.89.

- [Rare facial phenotype identification on Face2Gene (test)](<https://www.sota2.com/research/sota/rare-facial-phenotype-identification-on-face2gene-test.md>): Rare facial phenotype identification; Face2Gene (test); R@5: 6.85.

- [Text-to-Image Retrieval on ZooClaw-Fashion short query (test)](<https://www.sota2.com/research/sota/text-to-image-retrieval-on-zooclaw-fashion-short-query-test.md>): Text-to-image fashion retrieval; ZooClaw-Fashion short query (test); R@1: 36.3.

- [Text-to-Image Retrieval on Fashion200k (R@10, MRR)](<https://www.sota2.com/research/sota/text-to-image-retrieval-on-fashion200k-r-at-10-mrr.md>): Text-to-image fashion retrieval; Fashion200k; R@10: 26.1.

- [Video Retrieval on DrivingDojo 14 (test)](<https://www.sota2.com/research/sota/video-retrieval-on-drivingdojo-14-test.md>): Video Retrieval; DrivingDojo 14 (test); MRR: 25.

- [Zero-shot Image-Text Retrieval on DOCCI](<https://www.sota2.com/research/sota/zero-shot-image-text-retrieval-on-docci.md>): Zero-shot Image-Text Retrieval; DOCCI; Accuracy: 62.2.

- [Depth Estimation on M3ED](<https://www.sota2.com/research/sota/depth-estimation-on-m3ed.md>): Depth Estimation; M3ED; Depth (m): 9.69.

- [Image Retrieval on COCO-Dense](<https://www.sota2.com/research/sota/image-retrieval-on-coco-dense.md>): Image Retrieval; COCO-Dense; Recall@1: 27.56.

- [Keypoints Detection on Soccer Pretraining Dataset](<https://www.sota2.com/research/sota/keypoints-detection-on-soccer-pretraining-dataset.md>): Keypoints Detection; Soccer Pretraining Dataset; Accuracy: 93.7.

- [Retrieval on MMLongBench Doc](<https://www.sota2.com/research/sota/retrieval-on-mmlongbench-doc.md>): Retrieval; MMLongBench-Doc; MRR@10: 42.02.

- [Robotic Manipulation on ManiSkill Franka Simulation 2](<https://www.sota2.com/research/sota/robotic-manipulation-on-maniskill-franka-simulation-2.md>): Robotic Manipulation; ManiSkill Franka Simulation 2; Lift Peg Success Rate: 58.33.

- [Structure-aware retrieval on DiagramBank](<https://www.sota2.com/research/sota/structure-aware-retrieval-on-diagrambank.md>): Structure-aware retrieval; DiagramBank; MRR: 45.3.

- [Structure-aware retrieval on FigureBench](<https://www.sota2.com/research/sota/structure-aware-retrieval-on-figurebench.md>): Structure-aware retrieval; FigureBench; MRR: 33.3.

- [Viewpoint Grounding on Sr3D (R@K)](<https://www.sota2.com/research/sota/viewpoint-grounding-on-sr3d-r-at-k.md>): Viewpoint Grounding; Sr3D; Recall@1: 13.

- [Classification on FMOW (Functional Map of the World) (test)](<https://www.sota2.com/research/sota/classification-on-fmow-functional-map-of-the-world-test.md>): Classification; FMOW (Functional Map of the World) (test); Top-1 Acc: 41.25.

- [Composition-based Retrieval on nuImages](<https://www.sota2.com/research/sota/composition-based-retrieval-on-nuimages.md>): Composition-based Retrieval; nuImages; Comp@1: 40.38.

- [Cross-modal Retrieval Suite MS-COCO & Flickr30K](<https://www.sota2.com/research/sota/cross-modal-retrieval-suite-ms-coco-and-flickr30k.md>): Cross-modal retrieval; Cross-modal Retrieval Suite MS-COCO & Flickr30K; Average Score: 76.7.

- [Image-to-Text Retrieval on NWPU](<https://www.sota2.com/research/sota/image-to-text-retrieval-on-nwpu.md>): Image-to-Text Retrieval; NWPU; Avg Top-1/5/10: 31.86.

- [Video Object Tracking on DAVIS 2017 (val)](<https://www.sota2.com/research/sota/video-object-tracking-on-davis-2017-val.md>): Video Object Tracking; DAVIS 2017 (val); J&F Score: 32.4.

- [Commentary Generation on SN-Caption (test-align)](<https://www.sota2.com/research/sota/commentary-generation-on-sn-caption-test-align.md>): Commentary Generation; SN-Caption (test-align); BLEU@1: 28.9.

- [Video Retrieval on PE-Video](<https://www.sota2.com/research/sota/video-retrieval-on-pe-video.md>): Video Retrieval; PE-Video; R@1: 47.55.

- [Video Retrieval on DREAM 1K](<https://www.sota2.com/research/sota/video-retrieval-on-dream-1k.md>): Video Retrieval; DREAM 1k; R@1: 61.5.

- [Video-Commentary Alignment on Soccer Pretraining Dataset](<https://www.sota2.com/research/sota/video-commentary-alignment-on-soccer-pretraining-dataset.md>): Video-Commentary Alignment; Soccer Pretraining Dataset; Top-1 Accuracy: 3.4.

- [Image Classification on Food-101](<https://www.sota2.com/research/sota/image-classification-on-food-101.md>): Image Classification; Food-101; Accuracy: 97.

- [Chart Question Answering on ChartQA (Accuracy)](<https://www.sota2.com/research/sota/chart-question-answering-on-chartqa-accuracy.md>): Chart Question Answering; ChartQA; Accuracy: 89.4.

- [Zero-shot Image Classification on ImageNet-1K](<https://www.sota2.com/research/sota/zero-shot-image-classification-on-imagenet-1k.md>): Zero-shot Image Classification; ImageNet-1K; Top-1 Accuracy: 78.5.

- [Image Classification on Oxford 102 Flowers](<https://www.sota2.com/research/sota/image-classification-on-oxford-102-flowers.md>): Image Classification; Oxford 102 Flowers; Top-1 Accuracy: 99.3.

- [Multimodal Reasoning on SEED-BENCH](<https://www.sota2.com/research/sota/multimodal-reasoning-on-seed-bench.md>): Multimodal Reasoning; SEED-Bench; Accuracy: 77.7.

- [Image Classification on ImageNet (val) (Accuracy and Throughput)](<https://www.sota2.com/research/sota/image-classification-on-imagenet-val-accuracy-and-throughput.md>): Image Classification; ImageNet (val); Accuracy (%): 84.1.

- [Image-Text Retrieval on MSCOCO (val)](<https://www.sota2.com/research/sota/image-text-retrieval-on-mscoco-val.md>): Image-Text Retrieval; MSCOCO (val); T2I Recall@1: 52.5.

- [Image Classification on Durian (test)](<https://www.sota2.com/research/sota/image-classification-on-durian-test.md>): Image Classification; Durian (test); Accuracy: 90.06.

- [Image Classification on Radish (test)](<https://www.sota2.com/research/sota/image-classification-on-radish-test.md>): Image Classification; Radish (test); Accuracy: 92.95.

- [5-way 1-shot Classification on CropDisease](<https://www.sota2.com/research/sota/5-way-1-shot-classification-on-cropdisease.md>): 5-way 1-shot Classification; CropDisease; Accuracy: 87.5.

- [Text-to-Image Retrieval on IIW](<https://www.sota2.com/research/sota/text-to-image-retrieval-on-iiw.md>): Text-to-Image Retrieval; IIW; Score: 95.6.

- [Action Recognition on Mimetics](<https://www.sota2.com/research/sota/action-recognition-on-mimetics.md>): Action Recognition; Mimetics; Accuracy: 48.95.

- [Classification on BUS-BRA (5-fold cross-validation)](<https://www.sota2.com/research/sota/classification-on-bus-bra-5-fold-cross-validation.md>): Classification; BUS-BRA (5-fold cross-validation); AUROC: 90.8.

- [Tactile Recognition on TAG→OF-Real (test)](<https://www.sota2.com/research/sota/tactile-recognition-on-tag-of-real-test.md>): Tactile Recognition; TAG→OF-Real (test); Accuracy: 56.

- [Multimodal Language Understanding on MMMU](<https://www.sota2.com/research/sota/multimodal-language-understanding-on-mmmu.md>): Multimodal Language Understanding; MMMU; Accuracy: 50.2.

- [Multimodal Understanding on LVBench](<https://www.sota2.com/research/sota/multimodal-understanding-on-lvbench.md>): Multimodal Understanding; LVBench; Accuracy: 57.

- [Multimodal Understanding on Aggregated Multimodal Suite](<https://www.sota2.com/research/sota/multimodal-understanding-on-aggregated-multimodal-suite.md>): Multimodal Understanding; Aggregated Multimodal Suite; Average Accuracy: 72.81.

- [Document and OCR on OCR-B OCRBench](<https://www.sota2.com/research/sota/document-and-ocr-on-ocr-b-ocrbench.md>): Document and OCR; OCR-B OCRBench; Accuracy Score: 47.3.

- [Document and OCR on SEED-Bench-2-Plus](<https://www.sota2.com/research/sota/document-and-ocr-on-seed-bench-2-plus.md>): Document and OCR; SEED-Bench-2-Plus; Accuracy Score: 50.3.

- [Holistic Multimodal Understanding on 14 Benchmarks Composite](<https://www.sota2.com/research/sota/holistic-multimodal-understanding-on-14-benchmarks-composite.md>): Holistic Multimodal Understanding; 14 Benchmarks Composite; Average Score: 61.5.

- [Document and OCR Understanding on Doc&OCR Suite](<https://www.sota2.com/research/sota/document-and-ocr-understanding-on-doc-and-ocr-suite.md>): Document and OCR Understanding; Doc&OCR Suite; ChartQA: 47.2.

- [Pick Fruits on Real-world Dexterous Manipulation Dexmate Vega](<https://www.sota2.com/research/sota/pick-fruits-on-real-world-dexterous-manipulation-dexmate-vega.md>): Pick Fruits; Real-world Dexterous Manipulation Dexmate Vega; Success Rate: 52.08.

- [Visual Question Answering on General VQA Suite](<https://www.sota2.com/research/sota/visual-question-answering-on-general-vqa-suite.md>): Visual Question Answering; General VQA Suite; SEED-2 Accuracy: 56.4.

- [Viewpoint Grounding on Locate-3D (ScanNet++)](<https://www.sota2.com/research/sota/viewpoint-grounding-on-locate-3d-scannet-plus-plus.md>): Viewpoint Grounding; Locate-3D (ScanNet++); Performance on Room Type Prompt: 47.12.

- [3D Awareness Probing (Geometry + Texture) on Feat2GS average of 6 datasets](<https://www.sota2.com/research/sota/3d-awareness-probing-geometry-plus-texture-on-feat2gs-average-of-6-datasets.md>): 3D Awareness Probing (Geometry + Texture); Feat2GS average of 6 datasets; PSNR: 19.36.

- [3D Awareness Probing (Geometry) on Feat2GS](<https://www.sota2.com/research/sota/3d-awareness-probing-geometry-on-feat2gs.md>): 3D Awareness Probing (Geometry); Feat2GS; PSNR: 19.39.

- [Classification on DermaMNIST Linear Evaluation 100% ratio](<https://www.sota2.com/research/sota/classification-on-dermamnist-linear-evaluation-100-percent-ratio.md>): Classification; DermaMNIST Linear Evaluation 100% ratio; Accuracy: 83.5.

- [Classification on RetinaMNIST Linear Evaluation 100% ratio](<https://www.sota2.com/research/sota/classification-on-retinamnist-linear-evaluation-100-percent-ratio.md>): Classification; RetinaMNIST Linear Evaluation 100% ratio; Accuracy: 61.

- [Classification on HAM10000 Linear Evaluation 10% ratio](<https://www.sota2.com/research/sota/classification-on-ham10000-linear-evaluation-10-percent-ratio.md>): Classification; HAM10000 Linear Evaluation 10% ratio; Accuracy: 66.67.

- [Multimodal Retrieval on RAVENEA (test)](<https://www.sota2.com/research/sota/multimodal-retrieval-on-ravenea-test.md>): Multimodal Retrieval; RAVENEA (test); MRR: 68.62.

- [Query-to-image retrieval on ChartGalaxy Synthetic Multi-facet 1.0 (test)](<https://www.sota2.com/research/sota/query-to-image-retrieval-on-chartgalaxy-synthetic-multi-facet-1-0-test.md>): Query-to-image retrieval; ChartGalaxy Synthetic Multi-facet 1.0 (test); R@1: 67.59.

- [Scene Classification on UCM (test)](<https://www.sota2.com/research/sota/scene-classification-on-ucm-test.md>): Scene Classification; UCM (test); Top-1 Acc: 81.43.

- [Text-to-Image Fashion Retrieval on H&M](<https://www.sota2.com/research/sota/text-to-image-fashion-retrieval-on-h-and-m.md>): Text-to-image fashion retrieval; H&M; R@10: 12.

- [Image Retrieval on Flickr30K-Dense](<https://www.sota2.com/research/sota/image-retrieval-on-flickr30k-dense.md>): Image Retrieval; Flickr30K-Dense; Recall@1: 5.12.

- [Open-vocabulary semantic segmentation on PASCAL VOC VOC20 2012](<https://www.sota2.com/research/sota/open-vocabulary-semantic-segmentation-on-pascal-voc-voc20-2012.md>): Open Vocabulary Semantic Segmentation; PASCAL VOC VOC20 2012; mIoU: 68.

- [Optical Flow Estimation on OctoSense degraded (test)](<https://www.sota2.com/research/sota/optical-flow-estimation-on-octosense-degraded-test.md>): Optical Flow Estimation; OctoSense degraded (test); Flow Error (px): 19.02.

- [Depth Estimation on NAVI](<https://www.sota2.com/research/sota/depth-estimation-on-navi.md>): Depth Estimation; NAVI; Depth Error: 0.064.

- [Image-to-Text Retrieval on UCM-Captions](<https://www.sota2.com/research/sota/image-to-text-retrieval-on-ucm-captions.md>): Image-to-Text Retrieval; UCM Captions; Average Recall (R@1, R@5, R@10): 70.48.

- [Image-to-Text Retrieval on GeoSound Bing imagery, scale=1](<https://www.sota2.com/research/sota/image-to-text-retrieval-on-geosound-bing-imagery-scale-1.md>): Image-to-Text Retrieval; GeoSound Bing imagery, scale=1; R@10%: 44.9.

- [Surface Normal Estimation on NAVI](<https://www.sota2.com/research/sota/surface-normal-estimation-on-navi.md>): Surface Normal Estimation; NAVI; Mean Angular Error: 25.

- [Text-to-Image Retrieval on GeoSound Bing imagery, scale=1](<https://www.sota2.com/research/sota/text-to-image-retrieval-on-geosound-bing-imagery-scale-1.md>): Text-to-Image Retrieval; GeoSound Bing imagery, scale=1; Recall@10: 39.7.

- [Image Captioning on SS v2](<https://www.sota2.com/research/sota/image-captioning-on-ss-v2.md>): Captioning; SS v2; CIDEr: 198.6.

- [Event Classification on Soccer Pretraining Dataset](<https://www.sota2.com/research/sota/event-classification-on-soccer-pretraining-dataset.md>): Event Classification; Soccer Pretraining Dataset; Accuracy: 0.498.

- [Image Classification on ImageNet-Sketch](<https://www.sota2.com/research/sota/image-classification-on-imagenet-sketch.md>): Image Classification; ImageNet-Sketch; Top-1 Accuracy: 81.

- [Image Classification on BreastMNIST](<https://www.sota2.com/research/sota/image-classification-on-breastmnist.md>): Image Classification; BreastMNIST; Accuracy: 89.1.

- [Image Classification on vtab pets](<https://www.sota2.com/research/sota/image-classification-on-vtab-pets.md>): Image Classification; vtab pets; Accuracy: 95.4.

- [Image-Text Retrieval on Flickr30k (val)](<https://www.sota2.com/research/sota/image-text-retrieval-on-flickr30k-val.md>): Image-Text Retrieval; Flickr30k (val); Text-to-Image Recall@1: 80.

- [Image Classification on Black gram (test)](<https://www.sota2.com/research/sota/image-classification-on-black-gram-test.md>): Image Classification; Black gram (test); Accuracy: 0.9003.

- [Image-to-Text Retrieval on PhenoBench](<https://www.sota2.com/research/sota/image-to-text-retrieval-on-phenobench.md>): Image-to-Text Retrieval; PhenoBench; Recall@10: 14.02.

- [Semantic Segmentation on PASCAL](<https://www.sota2.com/research/sota/semantic-segmentation-on-pascal.md>): Semantic segmentation; Pascal; mIoU: 78.1.

- [Image-Text Retrieval on IIW-FG sentence-level](<https://www.sota2.com/research/sota/image-text-retrieval-on-iiw-fg-sentence-level.md>): Image-Text Retrieval; IIW-FG sentence-level; T2I Recall@1: 37.

- [Video Classification on EK100 (test val)](<https://www.sota2.com/research/sota/video-classification-on-ek100-test-val.md>): Video Classification; EK100 (test val); Top-1 Acc: 6.1.

- [Image Classification on ImageNet-1K (Top-1 Acc)](<https://www.sota2.com/research/sota/image-classification-on-imagenet-1k-top-1-acc.md>): Image Classification; ImageNet-1K; Top-1 Accuracy: 81.91.

- [Image-to-Text Retrieval on IIW (It Is What it is)](<https://www.sota2.com/research/sota/image-to-text-retrieval-on-iiw-it-is-what-it-is.md>): Image-to-Text Retrieval; IIW (It Is What it is); R@1: 94.8.

- [Multimodal Understanding on LLaVA-NeXT Multimodal Understanding Suite](<https://www.sota2.com/research/sota/multimodal-understanding-on-llava-next-multimodal-understanding-suite.md>): Multimodal Understanding; LLaVA-NeXT Multimodal Understanding Suite; ChartQA Accuracy: 79.1.

- [Pull Tissue on Real-world Dexterous Manipulation Dexmate Vega](<https://www.sota2.com/research/sota/pull-tissue-on-real-world-dexterous-manipulation-dexmate-vega.md>): Pull Tissue; Real-world Dexterous Manipulation Dexmate Vega; Success Rate: 25.

- [Action Recognition on HAT Action Swap](<https://www.sota2.com/research/sota/action-recognition-on-hat-action-swap.md>): Action Recognition; HAT Action Swap; SHAcc: 25.46.

- [Regression on 27 downstream regression tasks (test)](<https://www.sota2.com/research/sota/regression-on-27-downstream-regression-tasks-test.md>): Regression; 27 downstream regression tasks (test); Health Score: 34.5.

- [Scene Retrieval on ScanRefer (n=5)](<https://www.sota2.com/research/sota/scene-retrieval-on-scanrefer-n-5.md>): Scene Retrieval; ScanRefer (n=5); Recall@1: 1.5.

- [Scene Retrieval on ScanRefer (n=10)](<https://www.sota2.com/research/sota/scene-retrieval-on-scanrefer-n-10.md>): Scene Retrieval; ScanRefer (n=10); Recall@1: 1.8.

- [Scene Retrieval on Nr3D n=5](<https://www.sota2.com/research/sota/scene-retrieval-on-nr3d-n-5.md>): Scene Retrieval; Nr3D n=5; R@1: 0.7.

- [Scene Retrieval on Nr3D n=10](<https://www.sota2.com/research/sota/scene-retrieval-on-nr3d-n-10.md>): Scene Retrieval; Nr3D n=10; R@1: 1.

- [Scene Type Classification on ScanNet v2 (test)](<https://www.sota2.com/research/sota/scene-type-classification-on-scannet-v2-test.md>): Scene type classification; ScanNet v2 (test); Accuracy (0-shot): 36.

- [Classification on BreastMNIST Linear Evaluation 1% ratio](<https://www.sota2.com/research/sota/classification-on-breastmnist-linear-evaluation-1-percent-ratio.md>): Classification; BreastMNIST Linear Evaluation 1% ratio; Accuracy: 32.5.

- [Classification on BreastMNIST 100% ratio Linear Evaluation](<https://www.sota2.com/research/sota/classification-on-breastmnist-100-percent-ratio-linear-evaluation.md>): Classification; BreastMNIST 100% ratio Linear Evaluation; Accuracy: 83.

- [Classification on ChestMNIST Linear Evaluation 100% ratio](<https://www.sota2.com/research/sota/classification-on-chestmnist-linear-evaluation-100-percent-ratio.md>): Classification; ChestMNIST Linear Evaluation 100% ratio; Accuracy: 53.8.

- [Classification on OCTMNIST Linear Evaluation 1% ratio](<https://www.sota2.com/research/sota/classification-on-octmnist-linear-evaluation-1-percent-ratio.md>): Classification; OCTMNIST Linear Evaluation 1% ratio; Accuracy: 67.

- [Classification on OCTMNIST Linear Evaluation 10% ratio](<https://www.sota2.com/research/sota/classification-on-octmnist-linear-evaluation-10-percent-ratio.md>): Classification; OCTMNIST Linear Evaluation 10% ratio; Accuracy: 72.9.

- [Classification on OCTMNIST Linear Evaluation 100% ratio](<https://www.sota2.com/research/sota/classification-on-octmnist-linear-evaluation-100-percent-ratio.md>): Classification; OCTMNIST Linear Evaluation 100% ratio; Accuracy: 72.5.

- [Disease Classification on EchoGround-MIMIC (test)](<https://www.sota2.com/research/sota/disease-classification-on-echoground-mimic-test.md>): Disease Classification; EchoGround-MIMIC (test); AUC: 78.1.

- [Image Classification on Ophthalmology](<https://www.sota2.com/research/sota/image-classification-on-ophthalmology.md>): Image Classification; Ophthalmology; Accuracy: 23.

- [Image-text Retrieval on EchoGround-MIMIC (test)](<https://www.sota2.com/research/sota/image-text-retrieval-on-echoground-mimic-test.md>): Image-Text Retrieval; EchoGround-MIMIC (test); Recall@5: 0.0198.

- [Zero-shot Image-Text Retrieval on Flickr30k](<https://www.sota2.com/research/sota/zero-shot-image-text-retrieval-on-flickr30k.md>): Zero-shot Image-Text Retrieval; Flickr30K; Accuracy (Zero-shot): 96.5.

- [Classification on SkyScript (test)](<https://www.sota2.com/research/sota/classification-on-skyscript-test.md>): Classification; SkyScript (test); Top-1 Accuracy: 65.94.

- [Ego-motion Estimation on OctoSense degraded (test)](<https://www.sota2.com/research/sota/ego-motion-estimation-on-octosense-degraded-test.md>): Ego-motion estimation; OctoSense degraded (test); Translation Error (m): 0.71.

- [Open-vocabulary semantic segmentation on COCO-Stuff 164k](<https://www.sota2.com/research/sota/open-vocabulary-semantic-segmentation-on-coco-stuff-164k.md>): Open Vocabulary Semantic Segmentation; COCO-Stuff 164k; mIoU: 16.8.

- [Optical Flow on M3ED](<https://www.sota2.com/research/sota/optical-flow-on-m3ed.md>): Optical Flow; M3ED; Flow Error (px): 5.43.

- [Segmentation on ADE150](<https://www.sota2.com/research/sota/segmentation-on-ade150.md>): Segmentation; ADE150; mIoU: 15.6.

- [Classification on DTD](<https://www.sota2.com/research/sota/classification-on-dtd.md>): Classification; DTD; Accuracy: 67.2.

- [Video Action Classification on Kinetics-400](<https://www.sota2.com/research/sota/video-action-classification-on-kinetics-400.md>): Video Action Classification; Kinetics-400; Top-1 Accuracy: 0.873.

- [Image Classification on PathMNIST](<https://www.sota2.com/research/sota/image-classification-on-pathmnist.md>): Image Classification; PathMNIST; Accuracy: 94.1.

- [Overclustering on COCO Things](<https://www.sota2.com/research/sota/overclustering-on-coco-things.md>): Overclustering; COCO Things; mIoU: 50.7.

- [Text-to-Video Retrieval on MSVD (zero-shot)](<https://www.sota2.com/research/sota/text-to-video-retrieval-on-msvd-zero-shot.md>): Text-to-Video Retrieval; MSVD zero-shot; Recall@1: 67.2.

- [Open-vocabulary semantic segmentation on ADE20K A-847](<https://www.sota2.com/research/sota/open-vocabulary-semantic-segmentation-on-ade20k-a-847.md>): Open Vocabulary Semantic Segmentation; ADE20K A-847; mIoU: 14.3.

- [Medical Image Classification on ChestMNIST](<https://www.sota2.com/research/sota/medical-image-classification-on-chestmnist.md>): Medical Image Classification; ChestMNIST; Accuracy: 94.8.

- [Tactile Recognition on Tactile Cross-Domain OF Real to X Unseen target domains](<https://www.sota2.com/research/sota/tactile-recognition-on-tactile-cross-domain-of-real-to-x-unseen-target-domains.md>): Tactile Recognition; Tactile Cross-Domain OF Real to X Unseen target domains; Average ACC: 51.4.

- [Bounding box classification on COCO 80](<https://www.sota2.com/research/sota/bounding-box-classification-on-coco-80.md>): Bounding-box classification; COCO-80; Top-1 Accuracy: 62.

- [Bounding box classification on LVIS 1203](<https://www.sota2.com/research/sota/bounding-box-classification-on-lvis-1203.md>): Bounding-box classification; LVIS 1203; Top-1 Accuracy: 31.4.

- [Image Classification on Guava (test)](<https://www.sota2.com/research/sota/image-classification-on-guava-test.md>): Image Classification; Guava (test); Accuracy: 75.

- [5-way 5-shot classification on CropDisease](<https://www.sota2.com/research/sota/5-way-5-shot-classification-on-cropdisease.md>): 5-way 5-shot Classification; CropDisease; Accuracy: 96.43.

- [Text-to-Image Retrieval on PhenoBench](<https://www.sota2.com/research/sota/text-to-image-retrieval-on-phenobench.md>): Text-to-Image Retrieval; PhenoBench; R@10: 10.33.

- [AI-generated image detection on ConImageGen IC→IF](<https://www.sota2.com/research/sota/ai-generated-image-detection-on-conimagegen-ic-if.md>): AI-generated image detection; ConImageGen IC→IF; Accuracy: 63.

- [AI-generated Image Detection on PicoBanana (test)](<https://www.sota2.com/research/sota/ai-generated-image-detection-on-picobanana-test.md>): AI-generated image detection; PicoBanana (test); Accuracy: 59.2.

- [Image Classification on Caltech-UCSD Birds 200](<https://www.sota2.com/research/sota/image-classification-on-caltech-ucsd-birds-200.md>): Image Classification; Caltech-UCSD Birds 200; Accuracy: 77.8.

- [Video Classification on SS v2 (test val)](<https://www.sota2.com/research/sota/video-classification-on-ss-v2-test-val.md>): Video Classification; SS v2 (test val); Top-1 Accuracy: 6.1.

- [Image-to-Text Retrieval on Flickr-CNA (test)](<https://www.sota2.com/research/sota/image-to-text-retrieval-on-flickr-cna-test.md>): Image-to-Text Retrieval; Flickr-CNA (test); Recall@1: 79.8.

- [Image-to-Text Retrieval on COCO-CN (val)](<https://www.sota2.com/research/sota/image-to-text-retrieval-on-coco-cn-val.md>): Image-to-Text Retrieval; COCO-CN (val); Recall@1: 74.2.

- [Text-based Retrieval on MBE benchmark](<https://www.sota2.com/research/sota/text-based-retrieval-on-mbe-benchmark.md>): Text-based Retrieval; MBE benchmark; Recall@1: 3.64.

- [Text-based Retrieval on M5Product](<https://www.sota2.com/research/sota/text-based-retrieval-on-m5product.md>): Text-based Retrieval; M5Product; Recall@10: 13.3.

- [Text-to-Image Retrieval on MSCOCO (R@1, R@5, R@25, R@50)](<https://www.sota2.com/research/sota/text-to-image-retrieval-on-mscoco-r-at-1-r-at-5-r-at-25-r-at-50.md>): Text-to-Image Retrieval; MSCOCO; Recall@1: 49.28.

- [Classification on ChestMNIST Linear Evaluation 1% ratio](<https://www.sota2.com/research/sota/classification-on-chestmnist-linear-evaluation-1-percent-ratio.md>): Classification; ChestMNIST Linear Evaluation 1% ratio; Accuracy: 51.05.

- [Classification on DermaMNIST Linear Evaluation 1% ratio](<https://www.sota2.com/research/sota/classification-on-dermamnist-linear-evaluation-1-percent-ratio.md>): Classification; DermaMNIST Linear Evaluation 1% ratio; Accuracy: 69.

- [Classification on HAM10000 Linear Evaluation 1% ratio](<https://www.sota2.com/research/sota/classification-on-ham10000-linear-evaluation-1-percent-ratio.md>): Classification; HAM10000 Linear Evaluation 1% ratio; Accuracy: 53.24.

- [Image Classification on Hematology](<https://www.sota2.com/research/sota/image-classification-on-hematology.md>): Image Classification; Hematology; Accuracy: 8.3.

- [Query-to-image retrieval on ChartGalaxy Human Short 1.0 (test)](<https://www.sota2.com/research/sota/query-to-image-retrieval-on-chartgalaxy-human-short-1-0-test.md>): Query-to-image retrieval; ChartGalaxy Human Short 1.0 (test); Recall@1: 33.

- [Video Retrieval on MM-AU 38 (test)](<https://www.sota2.com/research/sota/video-retrieval-on-mm-au-38-test.md>): Video Retrieval; MM-AU 38 (test); MRR: 17.8.

- [Video Retrieval on CarCrashDataset 37 (test)](<https://www.sota2.com/research/sota/video-retrieval-on-carcrashdataset-37-test.md>): Video Retrieval; CarCrashDataset 37 (test); MRR: 28.9.

- [Ego-motion Estimation on M3ED](<https://www.sota2.com/research/sota/ego-motion-estimation-on-m3ed.md>): Ego-motion estimation; M3ED; Translation Error (m): 0.7.

- [Retrieval on MMLongBench Finreport](<https://www.sota2.com/research/sota/retrieval-on-mmlongbench-finreport.md>): Retrieval; MMLongBench Finreport; MRR@10: 1.45.

- [Retrieval on REAL-MM-RAG Finslides](<https://www.sota2.com/research/sota/retrieval-on-real-mm-rag-finslides.md>): Retrieval; REAL-MM-RAG Finslides; MRR@10: 4.93.

- [Retrieval on REAL-MM-RAG (Techreport)](<https://www.sota2.com/research/sota/retrieval-on-real-mm-rag-techreport.md>): Retrieval; REAL-MM-RAG (Techreport); MRR@10: 3.03.

- [Retrieval on REAL-MM-RAG Techslides](<https://www.sota2.com/research/sota/retrieval-on-real-mm-rag-techslides.md>): Retrieval; REAL-MM-RAG Techslides; MRR@10: 6.78.

- [Retrieval on LongDocURL Filtered](<https://www.sota2.com/research/sota/retrieval-on-longdocurl-filtered.md>): Retrieval; LongDocURL (Filtered); MRR@10: 5.43.

- [Semantic Segmentation on COCO Object](<https://www.sota2.com/research/sota/semantic-segmentation-on-coco-object.md>): Semantic segmentation; COCO Object; mIoU: 54.58.

- [Fine-grained Understanding on FG-OVD](<https://www.sota2.com/research/sota/fine-grained-understanding-on-fg-ovd.md>): Fine-grained Understanding; FG-OVD; Top-1 Accuracy (Hard): 26.

- [Local Image Retrieval (Image-to-Image) on ILIAS](<https://www.sota2.com/research/sota/local-image-retrieval-image-to-image-on-ilias.md>): Local Image Retrieval (Image-to-Image); ILIAS; mAP@50: 0.554.

- [Image Classification on Spinach (test)](<https://www.sota2.com/research/sota/image-classification-on-spinach-test.md>): Image Classification; Spinach (test); Accuracy: 54.6.

- [5-way 5-shot Classification on ISIC](<https://www.sota2.com/research/sota/5-way-5-shot-classification-on-isic.md>): 5-way 5-shot Classification; ISIC; Accuracy: 51.79.

- [5-way 5-shot Classification on EuroSAT](<https://www.sota2.com/research/sota/5-way-5-shot-classification-on-eurosat.md>): 5-way 5-shot Classification; EuroSAT; Accuracy: 91.39.

- [AI-generated image detection on ConImageGen IF→IC](<https://www.sota2.com/research/sota/ai-generated-image-detection-on-conimagegen-if-ic.md>): AI-generated image detection; ConImageGen IF→IC; Accuracy: 73.1.

- [Visual Document Retrieval on PlotQA](<https://www.sota2.com/research/sota/visual-document-retrieval-on-plotqa.md>): Visual document retrieval; PlotQA; NDCG@5: 38.

- [Classification on Resisc45 (test)](<https://www.sota2.com/research/sota/classification-on-resisc45-test.md>): Classification; Resisc45 (test); Top-1 Acc: 0.724.

- [Visual Question Answering on cVQA](<https://www.sota2.com/research/sota/visual-question-answering-on-cvqa.md>): Visual Question Answering; CVQA; Accuracy: 82.8.

- [Classification on m-pv ger 4](<https://www.sota2.com/research/sota/classification-on-m-pv-ger-4.md>): Classification; m-pv ger 4; Overall Accuracy (OA): 95.7.

- [Multi-label Object Classification on Pascal VOC 2007](<https://www.sota2.com/research/sota/multi-label-object-classification-on-pascal-voc-2007.md>): Multi-label Object Classification; PASCAL VOC 2007; Top-1 Accuracy: 77.

- [Semantic Segmentation on ADE20k (mIoU, mAcc)](<https://www.sota2.com/research/sota/semantic-segmentation-on-ade20k-miou-macc.md>): Semantic segmentation; ADE20K; mIoU: 45.6.

- [Pour on Real-world Dexterous Manipulation Dexmate Vega](<https://www.sota2.com/research/sota/pour-on-real-world-dexterous-manipulation-dexmate-vega.md>): Pour; Real-world Dexterous Manipulation Dexmate Vega; Success Rate: 35.42.

- [3D Awareness on NAVI](<https://www.sota2.com/research/sota/3d-awareness-on-navi.md>): 3D Awareness; NAVI; Accuracy: 49.4.

- [3D Awareness on SPair](<https://www.sota2.com/research/sota/3d-awareness-on-spair.md>): 3D Awareness; SPair; Accuracy: 42.6.

- [Scene Retrieval on Sr3D (n=10)](<https://www.sota2.com/research/sota/scene-retrieval-on-sr3d-n-10.md>): Scene Retrieval; Sr3D (n=10); R@1: 0.1.

- [View Classification on Multi-vendor TTE dataset (downstream)](<https://www.sota2.com/research/sota/view-classification-on-multi-vendor-tte-dataset-downstream.md>): View Classification; Multi-vendor TTE dataset (downstream); Accuracy: 89.7.

- [Classification on RSNA Linear Evaluation 1% ratio](<https://www.sota2.com/research/sota/classification-on-rsna-linear-evaluation-1-percent-ratio.md>): Classification; RSNA Linear Evaluation 1% ratio; Accuracy: 54.55.

- [Classification on RSNA Linear Evaluation 10% ratio](<https://www.sota2.com/research/sota/classification-on-rsna-linear-evaluation-10-percent-ratio.md>): Classification; RSNA Linear Evaluation 10% ratio; Accuracy: 68.9.

- [Classification on RSNA Linear Evaluation 100% ratio](<https://www.sota2.com/research/sota/classification-on-rsna-linear-evaluation-100-percent-ratio.md>): Classification; RSNA Linear Evaluation 100% ratio; Accuracy: 77.5.

- [Classification on BreastMNIST Linear Evaluation 10% ratio](<https://www.sota2.com/research/sota/classification-on-breastmnist-linear-evaluation-10-percent-ratio.md>): Classification; BreastMNIST Linear Evaluation 10% ratio; Accuracy: 73.5.

- [Classification on ChestMNIST Linear Evaluation 10% ratio](<https://www.sota2.com/research/sota/classification-on-chestmnist-linear-evaluation-10-percent-ratio.md>): Classification; ChestMNIST Linear Evaluation 10% ratio; Accuracy: 52.5.

- [Classification on RetinaMNIST Linear Evaluation 10% ratio](<https://www.sota2.com/research/sota/classification-on-retinamnist-linear-evaluation-10-percent-ratio.md>): Classification; RetinaMNIST Linear Evaluation 10% ratio; Accuracy: 54.5.

- [Classification on HAM10000 Linear Evaluation 100% ratio](<https://www.sota2.com/research/sota/classification-on-ham10000-linear-evaluation-100-percent-ratio.md>): Classification; HAM10000 Linear Evaluation 100% ratio; Accuracy: 72.57.

- [Image Classification on Dermatology](<https://www.sota2.com/research/sota/image-classification-on-dermatology.md>): Image Classification; Dermatology; Accuracy: 11.31.

- [Image Classification on Pathology](<https://www.sota2.com/research/sota/image-classification-on-pathology.md>): Image Classification; Pathology; Accuracy: 20.

- [Image Classification on Radiology](<https://www.sota2.com/research/sota/image-classification-on-radiology.md>): Image Classification; Radiology; Accuracy: 4.95.

- [Image Classification on PhenoBench](<https://www.sota2.com/research/sota/image-classification-on-phenobench.md>): Image Classification; PhenoBench; Accuracy: 0.25.

- [Query-to-image retrieval on ChartGalaxy Synthetic General 1.0 (test)](<https://www.sota2.com/research/sota/query-to-image-retrieval-on-chartgalaxy-synthetic-general-1-0-test.md>): Query-to-image retrieval; ChartGalaxy Synthetic General 1.0 (test); R@1: 19.38.

- [Query-to-image retrieval on ChartGalaxy Human Long 1.0 (test)](<https://www.sota2.com/research/sota/query-to-image-retrieval-on-chartgalaxy-human-long-1-0-test.md>): Query-to-image retrieval; ChartGalaxy Human Long 1.0 (test); R@1: 20.67.

- [Text-to-image fashion retrieval on ZooClaw-Fashion long query (test)](<https://www.sota2.com/research/sota/text-to-image-fashion-retrieval-on-zooclaw-fashion-long-query-test.md>): Text-to-image fashion retrieval; ZooClaw-Fashion long query (test); R@1: 32.2.

- [Text-to-Image Retrieval on XTD10 (test)](<https://www.sota2.com/research/sota/text-to-image-retrieval-on-xtd10-test.md>): Text-to-Image Retrieval; XTD10 (test); R@10: 83.4.

- [Image Classification on ImageNet A](<https://www.sota2.com/research/sota/image-classification-on-imagenet-a.md>): Image Classification; ImageNet A; Top-1 Acc: 90.5.

- [Image Classification on Food101 (Accuracy)](<https://www.sota2.com/research/sota/image-classification-on-food101-accuracy.md>): Image Classification; Food101; Accuracy: 94.7.

- [Image Captioning on TextCaps](<https://www.sota2.com/research/sota/image-captioning-on-textcaps.md>): Image Captioning; TextCaps; CIDEr: 134.5.

- [Infographic Visual Question Answering on InfoVQA](<https://www.sota2.com/research/sota/infographic-visual-question-answering-on-infovqa.md>): Infographic Visual Question Answering; InfoVQA; Accuracy: 77.8.

- [Fine-grained classification on Oxford Flowers 102](<https://www.sota2.com/research/sota/fine-grained-classification-on-oxford-flowers-102.md>): Fine grained classification; Oxford Flowers 102; Accuracy: 97.98.

- [Image-based Retrieval on M5Product](<https://www.sota2.com/research/sota/image-based-retrieval-on-m5product.md>): Image-based Retrieval; M5Product; Recall@10: 29.86.

- [Local Image Retrieval (Text-to-Image) on ILIAS](<https://www.sota2.com/research/sota/local-image-retrieval-text-to-image-on-ilias.md>): Local Image Retrieval (Text-to-Image); ILIAS; mAP@50: 45.2.

- [Image Classification on Eggplant (test)](<https://www.sota2.com/research/sota/image-classification-on-eggplant-test.md>): Image Classification; Eggplant (test); Accuracy: 0.443.

- [5-way 5-shot Classification on ChestX](<https://www.sota2.com/research/sota/5-way-5-shot-classification-on-chestx.md>): 5-way 5-shot Classification; ChestX; Accuracy: 24.24.

- [OCR on ChartQA](<https://www.sota2.com/research/sota/ocr-on-chartqa.md>): OCR; ChartQA; Score: 35.3.

- [Document Layout Analysis on DocLayNet](<https://www.sota2.com/research/sota/document-layout-analysis-on-doclaynet.md>): Document Layout Analysis; DocLayNet; mAP: 70.8.

- [Text-to-Image Retrieval on COCO-CN (val)](<https://www.sota2.com/research/sota/text-to-image-retrieval-on-coco-cn-val.md>): Text-to-Image Retrieval; COCO-CN (val); Recall@1: 51.7.

- [Attribute Prediction on MBE 2.0](<https://www.sota2.com/research/sota/attribute-prediction-on-mbe-2-0.md>): Attribute Prediction; MBE 2.0; Accuracy: 38.91.

- [Semantic Segmentation on CityScapes (mIoU, Accuracy)](<https://www.sota2.com/research/sota/semantic-segmentation-on-cityscapes-miou-accuracy.md>): Semantic segmentation; Cityscapes; mIoU: 63.4.

- [Text-to-Video Retrieval on MSR-VTT zero-shot](<https://www.sota2.com/research/sota/text-to-video-retrieval-on-msr-vtt-zero-shot.md>): Text-to-Video Retrieval; MSR-VTT zero-shot; R@1: 38.5.

- [Video Action Classification on Jester](<https://www.sota2.com/research/sota/video-action-classification-on-jester.md>): Video Action Classification; Jester; Top-1 Acc: 91.

- [Image Classification on Agri-Semantics 52k (test)](<https://www.sota2.com/research/sota/image-classification-on-agri-semantics-52k-test.md>): Image Classification; Agri-Semantics 52k (test); Overall Accuracy: 3.43.

- [k-NN Classification on Synthetic (test)](<https://www.sota2.com/research/sota/k-nn-classification-on-synthetic-test.md>): k-NN classification; Synthetic (test); Accuracy: 33.5.

- [Scene Retrieval on Sr3D n=5](<https://www.sota2.com/research/sota/scene-retrieval-on-sr3d-n-5.md>): Scene Retrieval; Sr3D n=5; R@1: 0.1.

- [Image Classification on ImageNet (Top-1 Acc)](<https://www.sota2.com/research/sota/image-classification-on-imagenet-top-1-acc.md>): Image Classification; ImageNet; Top-1 Accuracy: 85.

- [Image Classification on Country211](<https://www.sota2.com/research/sota/image-classification-on-country211.md>): Image Classification; Country211; Top-1 Acc: 73.6.

- [Classification on ImageNet (Accuracy)](<https://www.sota2.com/research/sota/classification-on-imagenet-accuracy.md>): Classification; ImageNet; Accuracy: 78.2.

- [Open Door on Meta-World](<https://www.sota2.com/research/sota/open-door-on-meta-world.md>): Open Door; Meta-World; VOC Score: 29.52.

- [Overclustering on Pascal VOC](<https://www.sota2.com/research/sota/overclustering-on-pascal-voc.md>): Overclustering; Pascal VOC; mIoU: 40.4.

- [Fine-grained Image Classification on Describable Textures Dataset (DTD)](<https://www.sota2.com/research/sota/fine-grained-image-classification-on-describable-textures-dataset-dtd.md>): Fine-grained Image Classification; Describable Textures Dataset (DTD); Top-1 Accuracy (Clean): 75.32.

- [Video Action Recognition on Perception Test](<https://www.sota2.com/research/sota/video-action-recognition-on-perception-test.md>): Video Action Recognition; Perception (test); Top-1 Accuracy: 53.3.

- [Video Action Recognition on CharadesEgo](<https://www.sota2.com/research/sota/video-action-recognition-on-charadesego.md>): Video Action Recognition; CharadesEgo; Top-1 Accuracy: 11.9.

- [5-way 1-shot Classification on ISIC](<https://www.sota2.com/research/sota/5-way-1-shot-classification-on-isic.md>): 5-way 1-shot Classification; ISIC; Accuracy: 33.47.

- [Geometric Correspondence on NAVI](<https://www.sota2.com/research/sota/geometric-correspondence-on-navi.md>): Geometric Correspondence; NAVI; Avg. Recall: 0.432.

- [AI-generated Image Detection on Cross-Dataset Average (PicoBanana & RAID) (test)](<https://www.sota2.com/research/sota/ai-generated-image-detection-on-cross-dataset-average-picobanana-and-raid-test.md>): AI-generated image detection; Cross-Dataset Average (PicoBanana & RAID) (test); Accuracy: 52.3.

- [Multimodal Retrieval (q^i -\> e^t) on MBE 3.0 1.0 (test)](<https://www.sota2.com/research/sota/multimodal-retrieval-q-i-e-t-on-mbe-3-0-1-0-test.md>): Multimodal Retrieval (q^i -\> e^t); MBE 3.0 1.0 (test); R@1: 1.1.

- [Multimodal Retrieval (q^t -\> e^i) on MBE 3.0 1.0 (test)](<https://www.sota2.com/research/sota/multimodal-retrieval-q-t-e-i-on-mbe-3-0-1-0-test.md>): Multimodal Retrieval (q^t -\> e^i); MBE 3.0 1.0 (test); R@1: 0.79.

- [Vision-Language Reasoning on ScanQA ScanNet scenes (test)](<https://www.sota2.com/research/sota/vision-language-reasoning-on-scanqa-scannet-scenes-test.md>): Vision-Language Reasoning; ScanQA ScanNet scenes (test); BLEU-1: 38.1.

- [Visual Document Retrieval on ChartQA](<https://www.sota2.com/research/sota/visual-document-retrieval-on-chartqa.md>): Visual document retrieval; ChartQA; NDCG@5: 69.9.

- [Action Recognition on EPIC-KITCHENS 100 (val)](<https://www.sota2.com/research/sota/action-recognition-on-epic-kitchens-100-val.md>): Action Recognition; Epic-Kitchens-100 (val); Top-1 Accuracy: 56.6.

- [Video Object Segmentation on DAVIS-S](<https://www.sota2.com/research/sota/video-object-segmentation-on-davis-s.md>): Video Object Segmentation; DAVIS S; J & F-Mean: 56.1.

- [Video Object Segmentation on YouTube-VOS S](<https://www.sota2.com/research/sota/video-object-segmentation-on-youtube-vos-s.md>): Video Object Segmentation; YouTube-VOS S; J & F-Mean: 52.

- [Visual Document Retrieval on ArXivQA](<https://www.sota2.com/research/sota/visual-document-retrieval-on-arxivqa.md>): Visual document retrieval; ArxiVQA; NDCG@5: 37.2.

- [Image-to-Text Retrieval on LIT-CN](<https://www.sota2.com/research/sota/image-to-text-retrieval-on-lit-cn.md>): Image-to-Text Retrieval; LIT-CN; Recall@1: 17.

- [Image-to-Text Retrieval on DCI-CN](<https://www.sota2.com/research/sota/image-to-text-retrieval-on-dci-cn.md>): Image-to-Text Retrieval; DCI-CN; Recall@1: 13.6.

- [Image-to-Text Retrieval on DOCCI-CN](<https://www.sota2.com/research/sota/image-to-text-retrieval-on-docci-cn.md>): Image-to-Text Retrieval; DOCCI-CN; Recall@1: 25.

- [Text-to-Image Retrieval on LIT-CN](<https://www.sota2.com/research/sota/text-to-image-retrieval-on-lit-cn.md>): Text-to-Image Retrieval; LIT-CN; Recall@1: 10.9.

- [Text-to-Image Retrieval on DCI-CN](<https://www.sota2.com/research/sota/text-to-image-retrieval-on-dci-cn.md>): Text-to-Image Retrieval; DCI-CN; Recall@1: 14.4.

- [Text-to-Image Retrieval on DOCCI-CN](<https://www.sota2.com/research/sota/text-to-image-retrieval-on-docci-cn.md>): Text-to-Image Retrieval; DOCCI-CN; Recall@1: 27.3.

- [Attribute Prediction on M5Product](<https://www.sota2.com/research/sota/attribute-prediction-on-m5product.md>): Attribute Prediction; M5Product; Accuracy: 43.01.

- [Image Recognition and Noise Filtering on Flooded-Car Study (val)](<https://www.sota2.com/research/sota/image-recognition-and-noise-filtering-on-flooded-car-study-val.md>): Image Recognition and Noise Filtering; Flooded-Car Study (val); Precision: 74.8.

- [Keypoint Matching on SPair-71K](<https://www.sota2.com/research/sota/keypoint-matching-on-spair-71k.md>): Keypoint Matching; SPair-71k; Accuracy (d=0): 9.29.

- [Material Selection on DuMaS](<https://www.sota2.com/research/sota/material-selection-on-dumas.md>): Material Selection; DuMaS; L1 Error: 53.1.

- [Product Classification on Our MBE](<https://www.sota2.com/research/sota/product-classification-on-our-mbe.md>): Product Classification; Our MBE; Accuracy: 17.85.

- [Action and Object Classification on SSv2, Diving-48, Jester, K400, COIN, IN1K Average](<https://www.sota2.com/research/sota/action-and-object-classification-on-ssv2-diving-48-jester-k400-coin-in1k-average.md>): Action and Object Classification; SSv2, Diving-48, Jester, K400, COIN, IN1K Average; Avg Accuracy: 81.1.

- [Fold Shorts on Real-world Dexterous Manipulation Dexmate Vega](<https://www.sota2.com/research/sota/fold-shorts-on-real-world-dexterous-manipulation-dexmate-vega.md>): Fold-Shorts; Real-world Dexterous Manipulation Dexmate Vega; Success Rate: 4.17.

- [Image-to-Text Retrieval on Flickr](<https://www.sota2.com/research/sota/image-to-text-retrieval-on-flickr.md>): Image-to-Text Retrieval; Flickr; R@1: 95.4.

- [Video Action Retrieval on SOVABench Inter-pair 1.0](<https://www.sota2.com/research/sota/video-action-retrieval-on-sovabench-inter-pair-1-0.md>): Video Action Retrieval; SOVABench Inter-pair 1.0; mAP: 30.6.

- [Monocular Depth Estimation on NYU v2 (val)](<https://www.sota2.com/research/sota/monocular-depth-estimation-on-nyu-v2-val.md>): Monocular Depth Estimation; NYU v2 (val); RMSE: 0.494.

- [5-way 1-shot Classification on EuroSAT](<https://www.sota2.com/research/sota/5-way-1-shot-classification-on-eurosat.md>): 5-way 1-shot Classification; EuroSAT; Accuracy: 74.16.

- [Video-to-Text Retrieval on Kinetics-400](<https://www.sota2.com/research/sota/video-to-text-retrieval-on-kinetics-400.md>): Video-to-Text retrieval; Kinetics-400; Top-1 Accuracy: 69.8.

- [Video-to-Text Retrieval on Kinetics-700](<https://www.sota2.com/research/sota/video-to-text-retrieval-on-kinetics-700.md>): Video-to-Text retrieval; Kinetics 700; Top-1 Accuracy: 61.8.

- [AI-generated video frame detection on GenVidBench frame-level samples](<https://www.sota2.com/research/sota/ai-generated-video-frame-detection-on-genvidbench-frame-level-samples.md>): AI-generated video frame detection; GenVidBench frame-level samples; CogVideo Accuracy: 50.5.

- [Average Classification Performance on Classification Multi-dataset Suite](<https://www.sota2.com/research/sota/average-classification-performance-on-classification-multi-dataset-suite.md>): Average Classification Performance; Classification Multi-dataset Suite; Average Class Score: 68.2.

- [Multimodal Retrieval (q^t -\> e^mm) on MBE 3.0 1.0 (test)](<https://www.sota2.com/research/sota/multimodal-retrieval-q-t-e-mm-on-mbe-3-0-1-0-test.md>): Multimodal Retrieval (q^t -\> e^mm); MBE 3.0 1.0 (test); Recall@1: 2.07.

- [Text-to-Image Retrieval on Flickr30k-CNA (test)](<https://www.sota2.com/research/sota/text-to-image-retrieval-on-flickr30k-cna-test.md>): Text-to-Image Retrieval; Flickr30k-CNA (test); R@1: 53.2.

- [Visual Document Retrieval on InfoVQA](<https://www.sota2.com/research/sota/visual-document-retrieval-on-infovqa.md>): Visual document retrieval; InfoVQA; NDCG@5: 41.6.

- [Visual Document Retrieval on DocVQA](<https://www.sota2.com/research/sota/visual-document-retrieval-on-docvqa.md>): Visual document retrieval; DocVQA; NDCG@5: 53.7.

- [Image-Text Retrieval on Urban 1k long-text](<https://www.sota2.com/research/sota/image-text-retrieval-on-urban-1k-long-text.md>): Image-Text Retrieval; Urban 1k long-text; T2I Recall@1: 57.8.

- [Product Classification on MBE 2.0](<https://www.sota2.com/research/sota/product-classification-on-mbe-2-0.md>): Product Classification; MBE 2.0; Accuracy: 11.18.

- [Image Classification on ImageNet-1K (Top-1 Accuracy)](<https://www.sota2.com/research/sota/image-classification-on-imagenet-1k-top-1-accuracy.md>): Image Classification; ImageNet-1K; Top-1 Acc: 88.

- [Video Action Recognition on HMDB51 (Top-1 Accuracy)](<https://www.sota2.com/research/sota/video-action-recognition-on-hmdb51-top-1-accuracy.md>): Video Action Recognition; HMDB51; Top-1 Accuracy: 81.2.

- [Image Classification on Oxford-IIIT Pet (Top-1 Acc)](<https://www.sota2.com/research/sota/image-classification-on-oxford-iiit-pet-top-1-acc.md>): Image Classification; Oxford-IIIT Pet; Top-1 Accuracy: 92.89.

- [Handwriting Retrieval on Handwriting Spanish synthetic disjoint fonts (Out-of-Domain (OOD))](<https://www.sota2.com/research/sota/handwriting-retrieval-on-handwriting-spanish-synthetic-disjoint-fonts-out-of-domain-ood.md>): Handwriting Retrieval; Handwriting Spanish synthetic disjoint fonts (Out-of-Domain (OOD)); Top-1 Accuracy: 55.26.

- [Multimodal Retrieval on M5Product](<https://www.sota2.com/research/sota/multimodal-retrieval-on-m5product.md>): Multimodal Retrieval; M5Product; Recall@1: 9.53.

- [Local Image Retrieval (Text-to-Image) on SORCE-1K](<https://www.sota2.com/research/sota/local-image-retrieval-text-to-image-on-sorce-1k.md>): Local Image Retrieval (Text-to-Image); SORCE-1K; Recall@1: 34.2.

- [Semantic Segmentation on OctoSense day (test)](<https://www.sota2.com/research/sota/semantic-segmentation-on-octosense-day-test.md>): Semantic segmentation; OctoSense day (test); mIoU: 37.7.

- [Diagnostic Classification on derm7pt Melanoma vs. Atypical Nevus (test)](<https://www.sota2.com/research/sota/diagnostic-classification-on-derm7pt-melanoma-vs-atypical-nevus-test.md>): Diagnostic Classification; derm7pt Melanoma vs. Atypical Nevus (test); Accuracy: 61.1.

- [Video-to-Text Retrieval on Kinetics-600](<https://www.sota2.com/research/sota/video-to-text-retrieval-on-kinetics-600.md>): Video-to-Text retrieval; Kinetics-600; Top-1 Accuracy: 67.

- [AI-generated Image Detection on RAID (test)](<https://www.sota2.com/research/sota/ai-generated-image-detection-on-raid-test.md>): AI-generated image detection; RAID (test); Accuracy: 45.4.

- [Average Retrieval Performance on Retrieval Multi-dataset Suite](<https://www.sota2.com/research/sota/average-retrieval-performance-on-retrieval-multi-dataset-suite.md>): Average Retrieval Performance; Retrieval Multi-dataset Suite; Recall@1: 49.4.

- [Geometric Understanding on ScanNet (test)](<https://www.sota2.com/research/sota/geometric-understanding-on-scannet-test.md>): Geometric Understanding; ScanNet scenes (test); RR@0.05m: 47.8.

- [Video-to-Text Retrieval on HMDB-51](<https://www.sota2.com/research/sota/video-to-text-retrieval-on-hmdb-51.md>): Video-to-Text retrieval; HMDB-51; Top-1 Accuracy: 51.8.

- [Vision-Language Reasoning on SQA3D ScanNet scenes (test)](<https://www.sota2.com/research/sota/vision-language-reasoning-on-sqa3d-scannet-scenes-test.md>): Vision-Language Reasoning; SQA3D ScanNet scenes (test); BLEU-1: 48.5.

- [Visual Grounding on ScanRefer (test, Overall Category)](<https://www.sota2.com/research/sota/visual-grounding-on-scanrefer-test-overall-category.md>): Visual Grounding; ScanRefer Overall category (test); Accuracy: 51.4.

- [Text-to-Image Retrieval on MIMIC-CXR (test)](<https://www.sota2.com/research/sota/text-to-image-retrieval-on-mimic-cxr-test.md>): Text-to-Image Retrieval; MIMIC-CXR (test); R@1: 0.27.

- [Image-to-Text Retrieval on COCO](<https://www.sota2.com/research/sota/image-to-text-retrieval-on-coco.md>): Image-to-Text Retrieval; COCO; R@1: 71.7.

- [Document Visual Question Answering on DocVQA (Accuracy)](<https://www.sota2.com/research/sota/document-visual-question-answering-on-docvqa-accuracy.md>): Document Visual Question Answering; DocVQA; Accuracy: 92.9.

- [Text-to-Image Retrieval on Flickr](<https://www.sota2.com/research/sota/text-to-image-retrieval-on-flickr.md>): Text-to-Image Retrieval; Flickr; R@1: 85.7.

- [Handwriting Retrieval on Handwriting In-Domain Set](<https://www.sota2.com/research/sota/handwriting-retrieval-on-handwriting-in-domain-set.md>): Handwriting Retrieval; Handwriting In-Domain Set; Accuracy@1: 67.34.

- [Robot Learning on CortexBench](<https://www.sota2.com/research/sota/robot-learning-on-cortexbench.md>): Robot Learning; CortexBench; Adroit Score: 56.5.

- [Image-based Retrieval on MBE benchmark](<https://www.sota2.com/research/sota/image-based-retrieval-on-mbe-benchmark.md>): Image-based Retrieval; MBE benchmark; Recall@1: 15.57.

- [Document and OCR on InfoVQA](<https://www.sota2.com/research/sota/document-and-ocr-on-infovqa.md>): Document and OCR; InfoVQA; Accuracy Score: 29.7.

- [Local Image Retrieval (Text-to-Image) on REIRCOCO](<https://www.sota2.com/research/sota/local-image-retrieval-text-to-image-on-reircoco.md>): Local Image Retrieval (Text-to-Image); REIRCOCO; Recall@1: 22.9.

- [Ego-motion Estimation on OctoSense night (test)](<https://www.sota2.com/research/sota/ego-motion-estimation-on-octosense-night-test.md>): Ego-motion estimation; OctoSense night (test); Translation RMSE (m): 0.78.

- [Diagnostic Classification on MIMIC-CXR Edema vs. Pneumonia (test)](<https://www.sota2.com/research/sota/diagnostic-classification-on-mimic-cxr-edema-vs-pneumonia-test.md>): Diagnostic Classification; MIMIC-CXR Edema vs. Pneumonia (test); Accuracy: 50.2.

- [Attribute Prediction on MBE 3.0 1.0 (test)](<https://www.sota2.com/research/sota/attribute-prediction-on-mbe-3-0-1-0-test.md>): Attribute Prediction; MBE 3.0 1.0 (test); Accuracy: 3.66.

- [Multimodal Retrieval on MBE 3.0 1.0 (test)](<https://www.sota2.com/research/sota/multimodal-retrieval-on-mbe-3-0-1-0-test.md>): Multimodal Retrieval (q^i -\> e^mm); MBE 3.0 1.0 (test); Recall@1: 5.31.

- [Video Classification on UCF101 (Top-1/Top-5 Accuracy)](<https://www.sota2.com/research/sota/video-classification-on-ucf101-top-1-top-5-accuracy.md>): Video Classification; UCF101; Top-1 Accuracy: 55.31.

- [Video-to-Text Retrieval on MSR-VTT (zero-shot)](<https://www.sota2.com/research/sota/video-to-text-retrieval-on-msr-vtt-zero-shot.md>): Video-to-Text retrieval; MSR-VTT zero-shot; R@1: 30.1.

- [Visual Document Retrieval on SlideVQA](<https://www.sota2.com/research/sota/visual-document-retrieval-on-slidevqa.md>): Visual document retrieval; SlideVQA; NDCG@5: 40.9.

- [Open Vocabulary Semantic Segmentation on Pascal VOC 20](<https://www.sota2.com/research/sota/open-vocabulary-semantic-segmentation-on-pascal-voc-20.md>): Open Vocabulary Semantic Segmentation; Pascal VOC 20; mIoU: 97.

- [Image Classification on Places 365](<https://www.sota2.com/research/sota/image-classification-on-places-365.md>): Image Classification; Places 365; Top-1 Acc: 56.18.

- [Monocular Depth Estimation on KITTI official (val)](<https://www.sota2.com/research/sota/monocular-depth-estimation-on-kitti-official-val.md>): Monocular Depth Estimation; KITTI official (val); RMSE: 3.273.

- [Image-Text Retrieval on ImageCoDe](<https://www.sota2.com/research/sota/image-text-retrieval-on-imagecode.md>): Image-Text Retrieval; ImageCoDe; R@1: 2.6.

- [Robot Manipulation on Franka Kitchen](<https://www.sota2.com/research/sota/robot-manipulation-on-franka-kitchen.md>): Robot Manipulation; Franka-Kitchen; Sliding Door Open Success: 58.5.

- [5-way 1-shot Classification on ChestX](<https://www.sota2.com/research/sota/5-way-1-shot-classification-on-chestx.md>): 5-way 1-shot Classification; ChestX; Accuracy (%): 21.44.

- [Visual Document Retrieval on ViDoRe 10 datasets v1](<https://www.sota2.com/research/sota/visual-document-retrieval-on-vidore-10-datasets-v1.md>): Visual document retrieval; ViDoRe 10 datasets v1; NDCG@5: 44.6.

- [Visual Document Retrieval on ViDoRe 8 datasets v3](<https://www.sota2.com/research/sota/visual-document-retrieval-on-vidore-8-datasets-v3.md>): Visual document retrieval; ViDoRe 8 datasets v3; NDCG@5: 14.8.

- [Multimodal Retrieval (q^mm -\> e^mm) on MBE 3.0 1.0 (test)](<https://www.sota2.com/research/sota/multimodal-retrieval-q-mm-e-mm-on-mbe-3-0-1-0-test.md>): Multimodal Retrieval (q^mm -\> e^mm); MBE 3.0 1.0 (test); Recall@1: 5.55.

- [Product Classification on MBE 3.0 1.0 (test)](<https://www.sota2.com/research/sota/product-classification-on-mbe-3-0-1-0-test.md>): Product Classification; MBE 3.0 1.0 (test); Accuracy: 1.47.

- [Product Classification on M5Product (test)](<https://www.sota2.com/research/sota/product-classification-on-m5product-test.md>): Product Classification; M5Product (test); Accuracy: 11.46.

- [Video Classification on Kinetics](<https://www.sota2.com/research/sota/video-classification-on-kinetics.md>): Video Classification; Kinetics; Top-1 Accuracy: 51.15.

- [Image Classification on ImageNet Robustness Suite](<https://www.sota2.com/research/sota/image-classification-on-imagenet-robustness-suite.md>): Image Classification; ImageNet Robustness Suite; Top-1 Accuracy (ImageNet-A): 84.3.

- [Video Captioning on VATEX (test)](<https://www.sota2.com/research/sota/video-captioning-on-vatex-test.md>): Video Captioning; VATEX (test); CIDEr: 58.4.

- [Image-to-Text Retrieval on COCO-CN](<https://www.sota2.com/research/sota/image-to-text-retrieval-on-coco-cn.md>): Image-to-Text Retrieval; COCO-CN; R@1: 72.

- [Action Recognition on K400](<https://www.sota2.com/research/sota/action-recognition-on-k400.md>): Action Recognition; K400; Top-1 Accuracy: 87.3.

- [Fine grained classification on Oxford-IIIT Pets](<https://www.sota2.com/research/sota/fine-grained-classification-on-oxford-iiit-pets.md>): Fine grained classification; Oxford-IIIT Pets; Top-1 Accuracy: 92.89.

- [Depth Prediction on OctoSense day (test)](<https://www.sota2.com/research/sota/depth-prediction-on-octosense-day-test.md>): Depth Prediction; OctoSense day (test); RMSE (m): 7.22.

- [Optical Flow Estimation on OctoSense night (test)](<https://www.sota2.com/research/sota/optical-flow-estimation-on-octosense-night-test.md>): Optical Flow Estimation; OctoSense night (test); EPE (px): 18.12.

- [Open-vocabulary semantic segmentation on ADE20K](<https://www.sota2.com/research/sota/open-vocabulary-semantic-segmentation-on-ade20k.md>): Open Vocabulary Semantic Segmentation; ADE20K; mIoU: 24.9.

- [Classification on Food101](<https://www.sota2.com/research/sota/classification-on-food101.md>): Classification; Food101; Top-1 Accuracy: 92.8.

- [Grasp Success Prediction on Grasp dataset](<https://www.sota2.com/research/sota/grasp-success-prediction-on-grasp-dataset.md>): Grasp Success Prediction; Grasp dataset; Accuracy: 56.4.

- [Image-Text Retrieval on DocVQA](<https://www.sota2.com/research/sota/image-text-retrieval-on-docvqa.md>): Image-Text Retrieval; DocVQA; R@1: 15.5.

- [Image-to-Text Retrieval on Fashion200K](<https://www.sota2.com/research/sota/image-to-text-retrieval-on-fashion200k.md>): Image-to-Text Retrieval; Fashion200k; R@5: 10.49.

- [Depth Prediction on OctoSense night (test)](<https://www.sota2.com/research/sota/depth-prediction-on-octosense-night-test.md>): Depth Prediction; OctoSense night (test); RMSE (m): 10.36.

- [Optical Flow Estimation on OctoSense day (test)](<https://www.sota2.com/research/sota/optical-flow-estimation-on-octosense-day-test.md>): Optical Flow Estimation; OctoSense day (test); EPE (px): 20.22.

- [Image Classification on Stanford Cars](<https://www.sota2.com/research/sota/image-classification-on-stanford-cars.md>): Image Classification; Stanford Cars; Accuracy: 95.9.

- [Fine-grained classification on Stanford Cars](<https://www.sota2.com/research/sota/fine-grained-classification-on-stanford-cars.md>): Fine grained classification; Stanford Cars; Accuracy: 92.55.

- [Multimodal Retrieval on MBE 2.0](<https://www.sota2.com/research/sota/multimodal-retrieval-on-mbe-2-0.md>): Multimodal Retrieval (text query to multimodal candidate); MBE 2.0; R@1: 23.58.

- [Chart Question Answering on ChartQA (Accuracy, Rel. (%))](<https://www.sota2.com/research/sota/chart-question-answering-on-chartqa-accuracy-rel-percent.md>): Chart Question Answering; ChartQA; Accuracy: 72.8.

- [Reward Modeling on Meta-World Open door](<https://www.sota2.com/research/sota/reward-modeling-on-meta-world-open-door.md>): Reward Modeling; Meta-World Open door; Prediction Accuracy: 57.27.

- [Image Query Multimodal Retrieval on M5Product (test)](<https://www.sota2.com/research/sota/image-query-multimodal-retrieval-on-m5product-test.md>): Multimodal Retrieval (image query to multimodal content); M5Product (test); Recall@1: 8.09.

- [Segmentation on m-chesapeake](<https://www.sota2.com/research/sota/segmentation-on-m-chesapeake.md>): Segmentation; m-chesapeake; Mean mIoU: 60.9.

- [Open Vocabulary Semantic Segmentation on PASCAL Context 59](<https://www.sota2.com/research/sota/open-vocabulary-semantic-segmentation-on-pascal-context-59.md>): Open Vocabulary Semantic Segmentation; Pascal Context 59; mIoU: 24.2.

- [Open-vocabulary semantic segmentation on ADE20K (val)](<https://www.sota2.com/research/sota/open-vocabulary-semantic-segmentation-on-ade20k-val.md>): Open Vocabulary Semantic Segmentation; ADE20K (val); mIoU: 10.8.

- [Semantic Segmentation on PC60](<https://www.sota2.com/research/sota/semantic-segmentation-on-pc60.md>): Semantic segmentation; PC60; mIoU: 0.196.

- [Ego-motion Estimation on OctoSense day (test)](<https://www.sota2.com/research/sota/ego-motion-estimation-on-octosense-day-test.md>): Ego-motion estimation; OctoSense day (test); Translation RMSE (m): 0.98.

- [Open-vocabulary semantic segmentation on Cityscapes](<https://www.sota2.com/research/sota/open-vocabulary-semantic-segmentation-on-cityscapes.md>): Open Vocabulary Semantic Segmentation; Cityscapes; mIoU: 39.7.

- [General Visual Question Answering on VQA v2 (Accuracy)](<https://www.sota2.com/research/sota/general-visual-question-answering-on-vqa-v2-accuracy.md>): General Visual Question Answering; VQA v2; Accuracy: 50.1.

- [Scene Recognition on Places-365](<https://www.sota2.com/research/sota/scene-recognition-on-places-365.md>): Scene recognition; Places 365; Top-1 Acc: 56.18.

- [AI-generated image detection on ConImageGen](<https://www.sota2.com/research/sota/ai-generated-image-detection-on-conimagegen.md>): AI-generated image detection; ConImageGen; Average Accuracy: 68.

- [Multimodal Retrieval on M5Product (test)](<https://www.sota2.com/research/sota/multimodal-retrieval-on-m5product-test.md>): Multimodal Retrieval (text query to multimodal content); M5Product (test); Recall@1: 9.53.

- [Text-to-Image Retrieval on Fashion200K](<https://www.sota2.com/research/sota/text-to-image-retrieval-on-fashion200k.md>): Text-to-Image Retrieval; Fashion200k; Recall@5: 10.03.

- [Video Action Retrieval on SOVABench Intra-pair 1.0](<https://www.sota2.com/research/sota/video-action-retrieval-on-sovabench-intra-pair-1-0.md>): Video Action Retrieval; SOVABench Intra-pair 1.0; Pair-mAP: 51.1.

- [Classification on CIFAR-100](<https://www.sota2.com/research/sota/classification-on-cifar-100.md>): Classification; CIFAR-100; Top-1 Accuracy: 76.9.

- [Document Understanding on AI2D](<https://www.sota2.com/research/sota/document-understanding-on-ai2d.md>): Document Understanding; AI2D; Accuracy: 0.67.

- [OCR-related understanding on DocVQA](<https://www.sota2.com/research/sota/ocr-related-understanding-on-docvqa.md>): OCR-related understanding; DocVQA; Score: 47.6.

- [Visual Document Retrieval on VisDoc OOD (NDCG@5)](<https://www.sota2.com/research/sota/visual-document-retrieval-on-visdoc-ood-ndcg-at-5.md>): Visual document retrieval; VisDoc OOD; NDCG@5: 34.8.

- [Visual Document Retrieval on IRPAPERS](<https://www.sota2.com/research/sota/visual-document-retrieval-on-irpapers.md>): Visual document retrieval; IRPAPERS; NDCG@5: 13.

- [Image-to-Text Retrieval on MIMIC-CXR (test)](<https://www.sota2.com/research/sota/image-to-text-retrieval-on-mimic-cxr-test.md>): Image-to-Text Retrieval; MIMIC-CXR (test); R@1: 0.08.

- [Product Classification on Fashion200K](<https://www.sota2.com/research/sota/product-classification-on-fashion200k.md>): Product Classification; Fashion200k; Accuracy: 12.18.

- [Vision Understanding on MMMU](<https://www.sota2.com/research/sota/vision-understanding-on-mmmu.md>): Vision Understanding; MMMU; Accuracy: 43.9.

- [Video Action Classification on HMDB51](<https://www.sota2.com/research/sota/video-action-classification-on-hmdb51.md>): Video Action Classification; HMDB51; Top-1 Accuracy: 47.83.

- [Video Classification on Kinetics-700](<https://www.sota2.com/research/sota/video-classification-on-kinetics-700.md>): Video Classification; Kinetics 700; Top-1 Accuracy: 61.8.

- [Few-shot Image Classification on CD-FSL 5-way 5-shot (test)](<https://www.sota2.com/research/sota/few-shot-image-classification-on-cd-fsl-5-way-5-shot-test.md>): Few-shot Image Classification; CD-FSL 5-way 5-shot (test); ChestX Accuracy: 24.24.

- [Classification on M5Product](<https://www.sota2.com/research/sota/classification-on-m5product.md>): Classification; M5Product; Accuracy: 13.22.

- [Text-to-image retrieval on MSCOCO](<https://www.sota2.com/research/sota/text-to-image-retrieval-on-mscoco.md>): Text-to-Image Retrieval; MSCOCO; R@1: 55.8.

- [General Visual Question Answering on GQA](<https://www.sota2.com/research/sota/general-visual-question-answering-on-gqa.md>): General Visual Question Answering; GQA; Accuracy: 45.2.

- [Button Press on Meta-World](<https://www.sota2.com/research/sota/button-press-on-meta-world.md>): Button press; Meta-World; VOC Score: -11.51.

- [Open Drawer on Meta-World](<https://www.sota2.com/research/sota/open-drawer-on-meta-world.md>): open drawer; Meta-World; VOC Score: 19.7.

- [Reward Modeling on Meta-World Button press](<https://www.sota2.com/research/sota/reward-modeling-on-meta-world-button-press.md>): Reward Modeling; Meta-World Button press; Prediction Accuracy: 45.15.

- [Reward Modeling on Meta-World Open drawer](<https://www.sota2.com/research/sota/reward-modeling-on-meta-world-open-drawer.md>): Reward Modeling; Meta-World Open drawer; Prediction Accuracy: 43.6.

- [Semantic Segmentation on PASCAL Context (mIoU, fwIoU, PACC)](<https://www.sota2.com/research/sota/semantic-segmentation-on-pascal-context-miou-fwiou-pacc.md>): Semantic segmentation; Pascal Context; mIoU: 47.51.

- [Image-to-Text Retrieval on ShareGPT4V](<https://www.sota2.com/research/sota/image-to-text-retrieval-on-sharegpt4v.md>): Image-to-Text Retrieval; ShareGPT4V; Recall@1: 85.1.

- [Optical Character Recognition on OCRBench (accuracy)](<https://www.sota2.com/research/sota/optical-character-recognition-on-ocrbench-accuracy.md>): Optical Character Recognition Benchmarking; OCRBench; Accuracy: 82.7.

- [Image Classification on ImageNet 1k (ACC)](<https://www.sota2.com/research/sota/image-classification-on-imagenet-1k-acc.md>): Image Classification; ImageNet-1K; Accuracy: 82.5.

- [Open-vocabulary semantic segmentation on Pascal Context 60](<https://www.sota2.com/research/sota/open-vocabulary-semantic-segmentation-on-pascal-context-60.md>): Open Vocabulary Semantic Segmentation; Pascal Context 60; mIoU: 32.4.

- [Video Action Classification on Diving-48](<https://www.sota2.com/research/sota/video-action-classification-on-diving-48.md>): Video Action Classification; Diving-48; Top-1 Acc: 75.3.

- [Few-shot Image Classification on CD-FSL 5-way 1-shot (test)](<https://www.sota2.com/research/sota/few-shot-image-classification-on-cd-fsl-5-way-1-shot-test.md>): Few-shot Image Classification; CD-FSL 5-way 1-shot (test); ChestX Accuracy: 21.44.

- [Text-to-Image Retrieval on COCO](<https://www.sota2.com/research/sota/text-to-image-retrieval-on-coco.md>): Text-to-Image Retrieval; COCO; Recall@1: 55.8.

- [5-way 5-shot Classification on CD-FSL Subset (ISIC, EuroSAT, CropDisease, ChestX) (test)](<https://www.sota2.com/research/sota/5-way-5-shot-classification-on-cd-fsl-subset-isic-eurosat-cropdisease-chestx-test.md>): 5-way 5-shot Classification; CD-FSL ISIC, EuroSAT, CropDisease, ChestX (test); Accuracy (ISIC): 51.79.

- [Image-to-text retrieval on MSCOCO](<https://www.sota2.com/research/sota/image-to-text-retrieval-on-mscoco.md>): Image-to-Text Retrieval; MSCOCO; R@1: 72.1.

- [Knowledge-based Visual Question Answering on OKVQA](<https://www.sota2.com/research/sota/knowledge-based-visual-question-answering-on-okvqa.md>): Knowledge-based Visual Question Answering; OKVQA; Accuracy: 0.5988.

- [Video-to-text retrieval on ActivityNet Captions](<https://www.sota2.com/research/sota/video-to-text-retrieval-on-activitynet-captions.md>): Video-to-Text retrieval; ActivityNet Captions; R@1: 33.4.

- [Open-vocabulary Segmentation on COCO Object](<https://www.sota2.com/research/sota/open-vocabulary-segmentation-on-coco-object.md>): Open-Vocabulary Segmentation; COCO Object; mIoU: 21.1.

- [Video-to-Text Retrieval on MSVD](<https://www.sota2.com/research/sota/video-to-text-retrieval-on-msvd.md>): Video-to-Text retrieval; MSVD; R@1: 74.6.

- [Web Agent Tasks on Mind2Web Cross-Domain](<https://www.sota2.com/research/sota/web-agent-tasks-on-mind2web-cross-domain.md>): Web agent tasks; Mind2Web Cross-Domain; Ele.Acc: 12.8.

- [Image-to-Text Retrieval on Flickr30k](<https://www.sota2.com/research/sota/image-to-text-retrieval-on-flickr30k.md>): Image-to-Text Retrieval; Flickr30K; R@1: 94.3.

- [Depth Estimation on NYU v2 (val)](<https://www.sota2.com/research/sota/depth-estimation-on-nyu-v2-val.md>): Depth Estimation; NYU v2 (val); RMSE: 0.5107.

- [Video-to-Text Retrieval on MSR-VTT 1K (test)](<https://www.sota2.com/research/sota/video-to-text-retrieval-on-msr-vtt-1k-test.md>): Video-to-Text retrieval; MSR-VTT 1K (test); R@1: 34.2.

- [Text-to-Video Retrieval on ActivityNet Captions](<https://www.sota2.com/research/sota/text-to-video-retrieval-on-activitynet-captions.md>): Text-to-Video Retrieval; ActivityNet Captions; R@1: 38.3.

- [Text-to-Image Retrieval on ShareGPT4V](<https://www.sota2.com/research/sota/text-to-image-retrieval-on-sharegpt4v.md>): Text-to-Image Retrieval; ShareGPT4V; R@1: 84.6.

- [Visual document retrieval on ViDoRe V2](<https://www.sota2.com/research/sota/visual-document-retrieval-on-vidore-v2.md>): Visual document retrieval; ViDoRe V2; Avg nDCG@5: 27.

- [General Visual Understanding on RealWorldQA](<https://www.sota2.com/research/sota/general-visual-understanding-on-realworldqa.md>): General Visual Understanding; RealworldQA; Accuracy: 58.7.

- [Classification on CIFAR-10](<https://www.sota2.com/research/sota/classification-on-cifar-10.md>): Classification; CIFAR-10; Accuracy: 94.8.

- [Text-to-Image Retrieval on COCO-CN](<https://www.sota2.com/research/sota/text-to-image-retrieval-on-coco-cn.md>): Text-to-Image Retrieval; COCO-CN; R@1: 50.7.

- [Open Vocabulary Semantic Segmentation on PASCAL VOC VOC21 with background 2012](<https://www.sota2.com/research/sota/open-vocabulary-semantic-segmentation-on-pascal-voc-voc21-with-background-2012.md>): Open Vocabulary Semantic Segmentation; PASCAL VOC VOC21 with background 2012; mIoU: 35.5.

- [Action Prediction on Mind2Web (Cross-Website)](<https://www.sota2.com/research/sota/action-prediction-on-mind2web-cross-website.md>): Web agent tasks; Mind2Web (Cross-Website); Element Accuracy: 11.8.

- [Diagram Question Answering on AI2D](<https://www.sota2.com/research/sota/diagram-question-answering-on-ai2d.md>): Diagram Question Answering; AI2D; AI2D Accuracy: 86.4.

- [Action Recognition on SSV2](<https://www.sota2.com/research/sota/action-recognition-on-ssv2.md>): Action Recognition; SSV2; Top-1 Acc: 58.2.

- [Image Classification on CIFAR100 (Baseline Accuracy)](<https://www.sota2.com/research/sota/image-classification-on-cifar100-baseline-accuracy.md>): Image Classification; CIFAR100; Accuracy: 91.2.

- [Text-to-Video Retrieval on MSR-VTT 1K (test)](<https://www.sota2.com/research/sota/text-to-video-retrieval-on-msr-vtt-1k-test.md>): Text-to-Video Retrieval; MSR-VTT 1K (test); R@1: 43.1.

- [Text-to-Video Retrieval on MSVD](<https://www.sota2.com/research/sota/text-to-video-retrieval-on-msvd.md>): Text-to-Video Retrieval; MSVD; R@1: 55.8.

- [Visual Question Answering on DocVQA (ANLS)](<https://www.sota2.com/research/sota/visual-question-answering-on-docvqa-anls.md>): Visual Question Answering; DocVQA; ANLS: 75.

- [Infographic Question Answering on InfoVQA](<https://www.sota2.com/research/sota/infographic-question-answering-on-infovqa.md>): Infographic Question Answering; InfoVQA; ANLS: 55.2.

- [Web Agent Tasks on Mind2Web Cross-Task](<https://www.sota2.com/research/sota/web-agent-tasks-on-mind2web-cross-task.md>): Web agent tasks; Mind2Web Cross-Task; Step Success Rate: 17.3.

- [Image Retrieval on Fashion200k (test)](<https://www.sota2.com/research/sota/image-retrieval-on-fashion200k-test.md>): Image Retrieval; Fashion200k (test); Recall@1: 4.39.

- [Open-Vocabulary Semantic Segmentation on Cityscapes (val)](<https://www.sota2.com/research/sota/open-vocabulary-semantic-segmentation-on-cityscapes-val.md>): Open Vocabulary Semantic Segmentation; Cityscapes (val); mIoU: 16.3.

- [Fine-Grained Visual Classification on FGVC Aircraft](<https://www.sota2.com/research/sota/fine-grained-visual-classification-on-fgvc-aircraft.md>): Fine-grained visual classification; FGVC Aircraft; Top-1 Accuracy: 62.8.

- [Image Retrieval on Flickr30K](<https://www.sota2.com/research/sota/image-retrieval-on-flickr30k.md>): Image Retrieval; Flickr30K; R@1: 83.72.

- [Action Recognition on Diving-48](<https://www.sota2.com/research/sota/action-recognition-on-diving-48.md>): Action Recognition; Diving-48; Top-1 Acc: 75.3.

- [5-way 1-shot Classification on CD-FSL Subset (test)](<https://www.sota2.com/research/sota/5-way-1-shot-classification-on-cd-fsl-subset-test.md>): 5-way 1-shot Classification; CD-FSL ISIC, EuroSAT, CropDisease, ChestX (test); Accuracy (ISIC): 33.47.

- [Classification on AID (test)](<https://www.sota2.com/research/sota/classification-on-aid-test.md>): Classification; AID (test); Top-1 Accuracy: 75.56.

- [Semantic Segmentation on PASCAL VOC](<https://www.sota2.com/research/sota/semantic-segmentation-on-pascal-voc.md>): Semantic segmentation; Pascal VOC; mIoU: 0.7541.

- [Image-to-Text Retrieval on DOCCI](<https://www.sota2.com/research/sota/image-to-text-retrieval-on-docci.md>): Image-to-Text Retrieval; DOCCI; R@1: 48.39.

- [Text-to-Image Retrieval on DOCCI](<https://www.sota2.com/research/sota/text-to-image-retrieval-on-docci.md>): Text-to-Image Retrieval; DOCCI; Recall@1: 49.24.

- [Single-needle retrieval on Visual Haystacks single-needle challenge](<https://www.sota2.com/research/sota/single-needle-retrieval-on-visual-haystacks-single-needle-challenge.md>): Single-needle retrieval; Visual Haystacks single-needle challenge; Recall@1: 72.

- [Text-to-Image Retrieval on Flickr30K](<https://www.sota2.com/research/sota/text-to-image-retrieval-on-flickr30k.md>): Text-to-Image Retrieval; Flickr30K; R@1: 82.6.

- [Text-to-Video Retrieval on VATEX](<https://www.sota2.com/research/sota/text-to-video-retrieval-on-vatex.md>): Text-to-Video Retrieval; VATEX; R@1: 67.5.

- [Image-to-Text Retrieval on Flickr30K-CN](<https://www.sota2.com/research/sota/image-to-text-retrieval-on-flickr30k-cn.md>): Image-to-Text Retrieval; Flickr30K-CN; R@1: 78.4.

- [Visual Question Answering on POPE](<https://www.sota2.com/research/sota/visual-question-answering-on-pope.md>): Visual Question Answering; POPE; Accuracy: 85.

- [Semantic Segmentation on COCO Stuff](<https://www.sota2.com/research/sota/semantic-segmentation-on-coco-stuff.md>): Semantic segmentation; COCO Stuff; mIoU: 38.7.

- [Video Action Classification on Something-Something v2](<https://www.sota2.com/research/sota/video-action-classification-on-something-something-v2.md>): Video Action Classification; Something-Something v2; Top-1 Acc: 49.9.

- [Compositional Reasoning on SugarCrepe](<https://www.sota2.com/research/sota/compositional-reasoning-on-sugarcrepe.md>): Compositional Reasoning; SugarCrepe; Overall Accuracy: 74.4.

- [Surface Normal Estimation on NYU v2](<https://www.sota2.com/research/sota/surface-normal-estimation-on-nyu-v2.md>): Surface Normal Estimation; NYU V2; Mean Angular Error: 23.

- [Multimodal Embedding on MMEB](<https://www.sota2.com/research/sota/multimodal-embedding-on-mmeb.md>): Multimodal Embedding; MMEB; Classification Accuracy: 40.3.

- [Video Classification on Kinetics-600](<https://www.sota2.com/research/sota/video-classification-on-kinetics-600.md>): Video Classification; Kinetics-600; Top-1 Accuracy: 67.

- [Image Classification on ImageNet V2](<https://www.sota2.com/research/sota/image-classification-on-imagenet-v2.md>): Image Classification; ImageNet V2; Top-1 Acc: 79.8.

- [Semantic Segmentation on PASCAL VOC 2012](<https://www.sota2.com/research/sota/semantic-segmentation-on-pascal-voc-2012.md>): Semantic segmentation; PASCAL VOC 2012; mIoU: 72.7.

- [Image Captioning on NoCaps](<https://www.sota2.com/research/sota/image-captioning-on-nocaps.md>): Image Captioning; NoCaps; CIDEr: 84.4.

- [Open Vocabulary Semantic Segmentation on Pascal Context PC-59](<https://www.sota2.com/research/sota/open-vocabulary-semantic-segmentation-on-pascal-context-pc-59.md>): Open Vocabulary Semantic Segmentation; Pascal Context PC-59; mIoU: 35.1.

- [Video-to-Text Retrieval on VATEX](<https://www.sota2.com/research/sota/video-to-text-retrieval-on-vatex.md>): Video-to-Text retrieval; VATEX; Recall@1: 68.2.

- [Text-to-Image Retrieval on DCI](<https://www.sota2.com/research/sota/text-to-image-retrieval-on-dci.md>): Text-to-Image Retrieval; DCI; R@1: 49.3.

- [Visual Question Answering on GQA](<https://www.sota2.com/research/sota/visual-question-answering-on-gqa.md>): Visual Question Answering; GQA; Accuracy: 65.2.

- [Video-to-Text Retrieval on DiDeMo](<https://www.sota2.com/research/sota/video-to-text-retrieval-on-didemo.md>): Video-to-Text retrieval; DiDeMo; R@1: 40.1.

- [Document Visual Question Answering on DocVQA](<https://www.sota2.com/research/sota/document-visual-question-answering-on-docvqa.md>): Document Visual Question Answering; DocVQA; ANLS: 90.2.

- [Image-to-Text Retrieval on DCI](<https://www.sota2.com/research/sota/image-to-text-retrieval-on-dci.md>): Image-to-Text Retrieval; DCI; R@1: 48.

- [Image Classification on ImageNet-1K (accuracy)](<https://www.sota2.com/research/sota/image-classification-on-imagenet-1k-accuracy.md>): Image Classification; ImageNet-1K; Accuracy: 78.2.

- [Text-to-Image Retrieval on Flickr30K-CN](<https://www.sota2.com/research/sota/text-to-image-retrieval-on-flickr30k-cn.md>): Text-to-Image Retrieval; Flickr30K-CN; R@1: 51.7.

- [Semantic Segmentation on VOC-20](<https://www.sota2.com/research/sota/semantic-segmentation-on-voc-20.md>): Semantic segmentation; VOC-20; mIoU: 73.

- [Semantic Segmentation on VOC21](<https://www.sota2.com/research/sota/semantic-segmentation-on-voc21.md>): Semantic segmentation; VOC21; mIoU: 26.8.

- [Video Action Recognition on Kinetics-400](<https://www.sota2.com/research/sota/video-action-recognition-on-kinetics-400.md>): Video Action Recognition; Kinetics-400; Top-1 Acc: 82.7.

- [Semantic Segmentation on ADE20K](<https://www.sota2.com/research/sota/semantic-segmentation-on-ade20k.md>): Semantic segmentation; ADE20K; mIoU: 51.6.

- [Visual Question Answering on ScienceQA](<https://www.sota2.com/research/sota/visual-question-answering-on-scienceqa.md>): Visual Question Answering; ScienceQA; Accuracy: 77.1.

- [Image Classification on ImageNet-ReaL](<https://www.sota2.com/research/sota/image-classification-on-imagenet-real.md>): Image Classification; ImageNet-ReaL; Precision@1: 88.5.

- [Depth Estimation on KITTI](<https://www.sota2.com/research/sota/depth-estimation-on-kitti.md>): Depth Estimation; KITTI; RMSE: 3.273.

- [Semantic Segmentation on Cityscapes (mean-IoU)](<https://www.sota2.com/research/sota/semantic-segmentation-on-cityscapes-mean-iou.md>): Semantic segmentation; Cityscapes; mIoU: 27.9.

- [Depth Estimation on NYU v2](<https://www.sota2.com/research/sota/depth-estimation-on-nyu-v2.md>): Depth Estimation; NYU V2; RMSE: 0.494.

- [Optical Character Recognition on OCRBench](<https://www.sota2.com/research/sota/optical-character-recognition-on-ocrbench.md>): Optical Character Recognition; OCRBench; Score: 671.

- [Diagram Understanding on AI2D](<https://www.sota2.com/research/sota/diagram-understanding-on-ai2d.md>): Diagram Understanding; AI2D; Accuracy: 79.5.

- [Visual Question Answering on TextVQA](<https://www.sota2.com/research/sota/visual-question-answering-on-textvqa.md>): Visual Question Answering; TextVQA; Accuracy: 74.3.

- [Semantic Segmentation on PC-59](<https://www.sota2.com/research/sota/semantic-segmentation-on-pc-59.md>): Semantic segmentation; PC-59; mIoU: 28.4.

- [Semantic Segmentation on ADE20K (mIoU)](<https://www.sota2.com/research/sota/semantic-segmentation-on-ade20k-miou.md>): Semantic segmentation; ADE20K; mIoU: 48.7.

- [Image Generation on ImageNet (val)](<https://www.sota2.com/research/sota/image-generation-on-imagenet-val.md>): Image Generation; ImageNet (val); Inception Score: 193.059.

- [Visual Question Answering on InfoVQA](<https://www.sota2.com/research/sota/visual-question-answering-on-infovqa.md>): Visual Question Answering; InfoVQA; Accuracy: 37.3.

- [Semantic Segmentation on PASCAL VOC (val)](<https://www.sota2.com/research/sota/semantic-segmentation-on-pascal-voc-val.md>): Semantic segmentation; PASCAL VOC (val); mIoU: 72.7.

- [Semantic Segmentation on Pascal VOC (default split)](<https://www.sota2.com/research/sota/semantic-segmentation-on-pascal-voc-default-split.md>): Semantic segmentation; Pascal VOC; mIoU: 79.1.

- [Text-to-Video Retrieval on MSR-VTT](<https://www.sota2.com/research/sota/text-to-video-retrieval-on-msr-vtt.md>): Text-to-Video Retrieval; MSR-VTT; Recall@1: 43.1.

- [Monocular Depth Estimation on NYU v2](<https://www.sota2.com/research/sota/monocular-depth-estimation-on-nyu-v2.md>): Monocular Depth Estimation; NYU V2; Delta 1 Acc: 82.

- [Visual Question Answering on AI2D](<https://www.sota2.com/research/sota/visual-question-answering-on-ai2d.md>): Visual Question Answering; AI2D; Accuracy: 73.6.

- [Multimodal Understanding on MMStar](<https://www.sota2.com/research/sota/multimodal-understanding-on-mmstar.md>): Multimodal Understanding; MMStar; Accuracy: 57.2.

- [Visual Question Answering on SimpleVQA (Accuracy)](<https://www.sota2.com/research/sota/visual-question-answering-on-simplevqa-accuracy.md>): Visual Question Answering; SimpleVQA; Accuracy: 0.285.

- [Video-to-Text Retrieval on MSR-VTT](<https://www.sota2.com/research/sota/video-to-text-retrieval-on-msr-vtt.md>): Video-to-Text retrieval; MSR-VTT; Recall@1: 34.3.

- [3D Semantic Segmentation on ScanNet v2 (val)](<https://www.sota2.com/research/sota/3d-semantic-segmentation-on-scannet-v2-val.md>): 3D Semantic Segmentation; ScanNet V2 (val); mIoU: 9.2.

- [Image Classification on ImageNet (Top-1 Accuracy)](<https://www.sota2.com/research/sota/image-classification-on-imagenet-top-1-accuracy.md>): Image Classification; ImageNet; Top-1 Accuracy: 69.87.

- [Fine-grained Image Classification on CUB-200 2011](<https://www.sota2.com/research/sota/fine-grained-image-classification-on-cub-200-2011.md>): Fine-grained Image Classification; CUB-200 2011; Accuracy: 78.41.

- [Visual Question Answering on RealworldQA](<https://www.sota2.com/research/sota/visual-question-answering-on-realworldqa.md>): Visual Question Answering; RealworldQA; Accuracy: 52.5.

- [Semantic Segmentation on Cityscapes (Semseg mIoU)](<https://www.sota2.com/research/sota/semantic-segmentation-on-cityscapes-semseg-miou.md>): Semantic segmentation; Cityscapes; mIoU: 59.53.

- [Multimodal Evaluation on MME](<https://www.sota2.com/research/sota/multimodal-evaluation-on-mme.md>): Multimodal Evaluation; MME; Score: 1730.

- [Semantic Segmentation on Cityscapes](<https://www.sota2.com/research/sota/semantic-segmentation-on-cityscapes.md>): Semantic segmentation; Cityscapes; mIoU: 64.8.

- [Visual Question Answering on ChartQA](<https://www.sota2.com/research/sota/visual-question-answering-on-chartqa.md>): Visual Question Answering; ChartQA; Accuracy: 72.8.

- [Text-to-Video Retrieval on DiDeMo](<https://www.sota2.com/research/sota/text-to-video-retrieval-on-didemo.md>): Text-to-Video Retrieval; DiDeMo; R@1: 0.392.

- [Image Classification on ImageNet-1k](<https://www.sota2.com/research/sota/image-classification-on-imagenet-1k.md>): Image Classification; ImageNet-1K; Top-1 Acc: 79.1.

- [Multimodal Understanding on SEED-Bench](<https://www.sota2.com/research/sota/multimodal-understanding-on-seed-bench.md>): Multimodal Understanding; SEED-Bench; Accuracy: 58.24.

- [Semantic Segmentation on Cityscapes (val) (mIoU only)](<https://www.sota2.com/research/sota/semantic-segmentation-on-cityscapes-val-miou-only.md>): Semantic segmentation; Cityscapes (val); mIoU: 64.8.

- [Action Recognition on Kinetics-400](<https://www.sota2.com/research/sota/action-recognition-on-kinetics-400.md>): Action Recognition; Kinetics-400; Top-1 Acc: 73.6.

- [Image Classification on ImageNet 1k (test)](<https://www.sota2.com/research/sota/image-classification-on-imagenet-1k-test.md>): Image Classification; ImageNet 1k (test); Top-1 Accuracy: 76.79.

- [Action Recognition on Something-Something v2 (val)](<https://www.sota2.com/research/sota/action-recognition-on-something-something-v2-val.md>): Action Recognition; Something-Something v2 (val); Top-1 Accuracy: 51.5.

- [Semantic Segmentation on ADE20K (val)](<https://www.sota2.com/research/sota/semantic-segmentation-on-ade20k-val.md>): Semantic segmentation; ADE20K (val); mIoU: 45.4.

- [Image Classification on CIFAR-10 (Accuracy)](<https://www.sota2.com/research/sota/image-classification-on-cifar-10-accuracy.md>): Image Classification; CIFAR-10; Accuracy: not recorded.

- [Image Classification on CIFAR-100 (ACC, NLL, ECE, ESS)](<https://www.sota2.com/research/sota/image-classification-on-cifar-100-acc-nll-ece-ess.md>): Image Classification; CIFAR-100; Accuracy: not recorded.

- [Image Classification on Oxford Flowers 102](<https://www.sota2.com/research/sota/image-classification-on-oxford-flowers-102.md>): Image Classification; Oxford Flowers 102; Accuracy: not recorded.

- [Text-to-Image Retrieval on MS-COCO](<https://www.sota2.com/research/sota/text-to-image-retrieval-on-ms-coco.md>): Text-to-Image Retrieval; MS-COCO; R@1: not recorded.

- [Depth Estimation on NYU Depth v2](<https://www.sota2.com/research/sota/depth-estimation-on-nyu-depth-v2.md>): Depth Estimation; NYU Depth V2; RMSE: not recorded.

- [Monocular Depth Estimation on KITTI](<https://www.sota2.com/research/sota/monocular-depth-estimation-on-kitti.md>): Monocular Depth Estimation; KITTI; Abs Rel: not recorded.

- [Image-to-Text Retrieval on MS-COCO](<https://www.sota2.com/research/sota/image-to-text-retrieval-on-ms-coco.md>): Image-to-Text Retrieval; MS-COCO; R@1: not recorded.

- [Video Classification on Kinetics 400](<https://www.sota2.com/research/sota/video-classification-on-kinetics-400.md>): Video Classification; Kinetics-400; Top-1 Acc: not recorded.

- [Image-Text Retrieval on RSICD](<https://www.sota2.com/research/sota/image-text-retrieval-on-rsicd.md>): Image-Text Retrieval; RSICD; Mean Recall: not recorded.

- [Image Classification on FGVC-Aircraft (Top-1 and Top-5 Accuracy)](<https://www.sota2.com/research/sota/image-classification-on-fgvc-aircraft-top-1-and-top-5-accuracy.md>): Image Classification; FGVC Aircraft; Top-1 Acc: not recorded.

- [Image Classification on ImageNet-A (OOD Evaluation)](<https://www.sota2.com/research/sota/image-classification-on-imagenet-a-ood-evaluation.md>): Image Classification; ImageNet A; Accuracy: not recorded.

- [Text-to-Image Retrieval on Urban-1K](<https://www.sota2.com/research/sota/text-to-image-retrieval-on-urban-1k.md>): Text-to-Image Retrieval; Urban-1K; R@1: not recorded.

- [Scene Recognition on SUN397](<https://www.sota2.com/research/sota/scene-recognition-on-sun397.md>): Scene recognition; SUN397; Accuracy: not recorded.

- [Image-to-text Retrieval on RSITMD](<https://www.sota2.com/research/sota/image-to-text-retrieval-on-rsitmd.md>): Image-to-Text Retrieval; RSITMD; Rank: not recorded.

- [Text-to-Image Retrieval on RSITMD](<https://www.sota2.com/research/sota/text-to-image-retrieval-on-rsitmd.md>): Text-to-Image Retrieval; RSITMD; mR: not recorded.

- [Image-Text Retrieval on DCI long-text](<https://www.sota2.com/research/sota/image-text-retrieval-on-dci-long-text.md>): Image-Text Retrieval; DCI long-text; Top-1 Accuracy: not recorded.

- [Visual Document Retrieval on ViDoRe Avg. across 4 datasets v2](<https://www.sota2.com/research/sota/visual-document-retrieval-on-vidore-avg-across-4-datasets-v2.md>): Visual document retrieval; ViDoRe Avg. across 4 datasets v2; Full NDCG: not recorded.

- [Multimodal Retrieval on Multi30k (test)](<https://www.sota2.com/research/sota/multimodal-retrieval-on-multi30k-test.md>): Multimodal Retrieval; Multi30K (test); Recall (EN): not recorded.

- [Image-to-Text Retrieval on Urban-1K](<https://www.sota2.com/research/sota/image-to-text-retrieval-on-urban-1k.md>): Image-to-Text Retrieval; Urban-1K; R@1: not recorded.

- [Semantic Correspondence on SPair-71k (PCK)](<https://www.sota2.com/research/sota/semantic-correspondence-on-spair-71k-pck.md>): Semantic Correspondence; SPair-71k; PCK@0.01: not recorded.

- [Action Recognition on Epic Kitchens 100](<https://www.sota2.com/research/sota/action-recognition-on-epic-kitchens-100.md>): Action Recognition; Epic Kitchens 100; Top-1 Acc: not recorded.

- [Visual document retrieval on ViDoRe V3](<https://www.sota2.com/research/sota/visual-document-retrieval-on-vidore-v3.md>): Visual document retrieval; ViDoRe V3; HR: not recorded.

## Original sources and implementation

[Original paper](<https://arxiv.org/abs/2502.14786>)

[PDF](<https://arxiv.org/pdf/2502.14786.pdf>)
