Multimodal Large Language Model Evaluation on MLLM Evaluation Suite
56.7Average Score (All)SigLIP 2 (giant) + PIVOT
Evaluation results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| SigLIP 2 (giant) + PIVOTSize=giant, # Params=1000M, # Samples seen=40B + 0.003B, Fine-tuning Protocol=PIVOT2025.10 | 56.7 | 68.5 | 54.7 | 54.2 | 49.3 | |
| SigLIP 2 + PIVOTSize=So400m, # Params=400M, # Samples seen=40B + 0.003B, Fine-tuning Protocol=PIVOT2025.10 | 55.6 | 68.1 | 53.9 | 52.4 | 48.1 | |
| SigLIP 2 (giant) + SFTSize=giant, # Params=1000M, # Samples seen=40B + 0.003B, Fine-tuning Protocol=SFT2025.10 | 55.4 | 67.4 | 52.8 | 53.1 | 48.5 | |
| SigLIP 2 + SFTSize=So400m, # Params=400M, # Samples seen=40B + 0.003B, Fine-tuning Protocol=SFT2025.10 | 54.6 | 66.9 | 52.2 | 51.7 | 47.7 | |
| SigLIP 2 (giant)Size=giant, # Params=1000M, # Samples seen=40B, Fine-tuning Protocol=None2025.10 | 53.9 | 66.5 | 50.8 | 51.9 | 46.4 | |
| SigLIP 1-So400m_PIVOT + ConvNeXt-XXL# Params=1.25B2025.10 | 53.6 | 67.3 | 48.5 | 52.5 | 46 | |
| SigLIP 1 + PIVOTSize=So400m, # Params=400M, # Samples seen=30B + 0.003B, Fine-tuning Protocol=PIVOT2025.10 | 53.2 | 67.7 | 46.8 | 51.7 | 46.6 | |
| SigLIP 2Size=So400m, # Params=400M, # Samples seen=40B, Fine-tuning Protocol=None2025.10 | 52.4 | 66.2 | 46.6 | 50.6 | 46.1 | |
| SigLIP 1 + SFTSize=So400m, # Params=400M, # Samples seen=30B + 0.003B, Fine-tuning Protocol=SFT2025.10 | 52.2 | 66.5 | 45.2 | 50.8 | 46.3 | |
| SigLIP 1-So400m + ConvNeXt-XXL# Params=1.25B2025.10 | 51.4 | 65.9 | 44.6 | 49.1 | 45.9 | |
| SigLIP 1Size=So400m, # Params=400M, # Samples seen=30B, Fine-tuning Protocol=None2025.10 | 50.9 | 65.4 | 42.3 | 49.8 | 46 | |
| CLIP + PIVOTSize=large, # Params=303M, # Samples seen=32B + 0.003B, Fine-tuning Protocol=PIVOT2025.10 | 49.5 | 64.6 | 37.8 | 48.6 | 47.1 | |
| SigLIP 1-So400m + DINOv2-L# Params=700M2025.10 | 49.4 | 64.5 | 41.5 | 46.5 | 45.1 | |
| CLIPSize=large, # Params=303M, # Samples seen=32B, Fine-tuning Protocol=None2025.10 | 46.3 | 62.1 | 35.1 | 43 | 45 | |
| DINOv2 + PIVOTSize=giant, # Params=1000M, # Samples seen=2B + 0.003B, Fine-tuning Protocol=PIVOT2025.10 | 43.6 | 62.1 | 18.7 | 49.2 | 44.3 | |
| DINOv2Size=giant, # Params=1000M, # Samples seen=2B, Fine-tuning Protocol=None2025.10 | 40.9 | 58.4 | 17.6 | 45.1 | 42.6 | |
| MAE + PIVOTSize=huge, # Params=632M, # Samples seen=2B + 0.003B, Fine-tuning Protocol=PIVOT2025.10 | 39.7 | 52.5 | 18.2 | 43.3 | 44.6 | |
| ImageNetSup + PIVOTSize=huge, # Params=632M, Fine-tuning Protocol=PIVOT2025.10 | 37.7 | 47.3 | 18.1 | 40.3 | 45.1 | |
| MOCO + PIVOTSize=base, # Params=86M, # Samples seen=1.4B + 0.003B, Fine-tuning Protocol=PIVOT2025.10 | 37.5 | 47.4 | 17.6 | 41 | 44.1 | |
| MAESize=huge, # Params=632M, # Samples seen=2B, Fine-tuning Protocol=None2025.10 | 36.8 | 47.6 | 17.3 | 40.2 | 42 | |
| ImageNetSupSize=huge, # Params=632M, # Samples seen=N/A, Fine-tuning Protocol=None2025.10 | 35.5 | 44.6 | 17.2 | 38.2 | 42.1 | |
| MOCOSize=base, # Params=86M, # Samples seen=1.4B, Fine-tuning Protocol=None2025.10 | 35.3 | 42.5 | 17.1 | 39.6 | 42.1 |