Image Classification on ImageNet-1K (Fine-tuning)
86.8Accuracy (FT)Softmax
Evaluation Results
| Method | Links | |
|---|---|---|
| SoftmaxBackbone=CLIP-L, Res.=512², Params (M)=304.20, FLOPS (G)=310.60, Peak Mem. (GB)=1.3181, Throughput (imgs/s)=36.52, fine-tuning=true2026.03 | 86.8 | |
| ViT-AdaLA (Ours)Backbone=CLIP-L, Res.=512², Params (M)=304.20, FLOPS (G)=262.19↓15.6%, Peak Mem. (GB)=1.2163↓7.7%, Throughput (imgs/s)=41.56↑16.1%, fine-tuning=true2026.03 | 85.5 | |
| MILANBackbone=ViT-B, Supervision=CLIP-B, Forward Ratio=25%, Reconstruct Ratio=100%, Epochs=4002022.11 | 85.4 | |
| MaskAlignBackbone=ViT-B, Supervision=CLIP-B, Forward Ratio=30%, Reconstruct Ratio=0%, Epochs=2002022.11 | 85.4 | |
| CMAEBackbone=CVIT-B, Supervision=RGB, Forward Ratio=25%, Reconstruct Ratio=75%, Epochs=16002022.11 | 85.3 | |
| MixMIMBackbone=MixMIM-B, Supervision=RGB, Forward Ratio=100%, Reconstruct Ratio=100%, Epochs=6002022.11 | 85.1 | |
| MCMAEBackbone=CVIT-B, Supervision=RGB, Forward Ratio=25%, Reconstruct Ratio=75%, Epochs=16002022.11 | 85 | |
| BEIT V2Backbone=ViT-B, Supervision=CLIP-B, Forward Ratio=100%, Reconstruct Ratio=40%, Epochs=3002022.11 | 85 | |
| MaskDistillBackbone=ViT-B, Supervision=CLIP-B, Forward Ratio=100%, Reconstruct Ratio=40%, Epochs=3002022.11 | 85 | |
| FD-CLIPBackbone=ViT-B, Supervision=CLIP-B, Forward Ratio=100%, Reconstruct Ratio=0%, Epochs=3002022.11 | 84.9 | |
| MVPBackbone=ViT-B, Supervision=CLIP-B, Forward Ratio=100%, Reconstruct Ratio=40%, Epochs=3002022.11 | 84.4 | |
| C2FMAEModel=ViT-B, Modality=RGB/Inst./Sem., Masking=Progressive, PT Epoch=1600, PT Cost=∼5.2×, Fine-tuning Input=multi-granular data, Input Size=224 × 224, Evaluation Protocol=Fine-tuning2026.03 | 84.4 | |
| data2vecBackbone=ViT-B, Supervision=EMA, Forward Ratio=100%, Reconstruct Ratio=60%, Epochs=8002022.11 | 84.2 | |
| C2FMAEModel=ViT-B, Modality=RGB/Inst./Sem., Masking=Progressive, PT Epoch=1600, PT Cost=∼5.2×, Input Size=224 × 224, Evaluation Protocol=Fine-tuning2026.03 | 84.2 | |
| MIRLModel=ViT-B, Modality=RGB, Masking=Random, PT Epoch=800, Input Size=224 × 224, Evaluation Protocol=Fine-tuning2026.03 | 84.1 | |
| SimMIMBackbone=Swin-B, Supervision=RGB, Forward Ratio=100%, Reconstruct Ratio=60%, Epochs=8002022.11 | 84 | |
| DMAEBackbone=ViT-B, Supervision=MAE-L, Forward Ratio=25%, Reconstruct Ratio=75%, Epochs=1002022.11 | 84 | |
| iBOTModel=ViT-B, Modality=RGB, Masking=Random, PT Epoch=1600, PT Cost=∼5.7×, Input Size=224 × 224, Evaluation Protocol=Fine-tuning2026.03 | 84 | |
| MaskFeatModel=ViT-B, Modality=RGB, Masking=Random, PT Epoch=1600, PT Cost=∼20.1×, Input Size=224 × 224, Evaluation Protocol=Fine-tuning2026.03 | 84 | |
| ROPIMModel=ViT-B, Modality=RGB, Masking=Random, PT Epoch=800, PT Cost=∼10.4×, Input Size=224 × 224, Evaluation Protocol=Fine-tuning2026.03 | 84 | |
| ConMIMModel=ViT-B, Modality=RGB, Masking=Random, PT Epoch=800, PT Cost=∼4.4×, Input Size=224 × 224, Evaluation Protocol=Fine-tuning2026.03 | 83.7 | |
| C2FMAEModel=ViT-B, Modality=RGB/Inst./Sem., Masking=Progressive, PT Epoch=400, PT Cost=∼1.3×, Input Size=224 × 224, Evaluation Protocol=Fine-tuning2026.03 | 83.7 | |
| MAEBackbone=ViT-B, Supervision=RGB, Forward Ratio=25%, Reconstruct Ratio=75%, Epochs=16002022.11 | 83.6 | |
| CAEBackbone=ViT-B, Supervision=DALLE, Forward Ratio=25%, Reconstruct Ratio=75%, Epochs=8002022.11 | 83.6 | |
| MaskFeatBackbone=ViT-B, Supervision=HOG, Forward Ratio=100%, Reconstruct Ratio=40%, Epochs=3002022.11 | 83.6 | |
| MAEModel=ViT-B, Modality=RGB, Masking=Random, PT Epoch=1600, PT Cost=∼4.0×, Input Size=224 × 224, Evaluation Protocol=Fine-tuning2026.03 | 83.6 | |
| CAEModel=ViT-B, Modality=RGB, Masking=Random, PT Epoch=800, PT Cost=∼4.6×, Input Size=224 × 224, Evaluation Protocol=Fine-tuning2026.03 | 83.6 | |
| ViT-AdaLA (Stage 2)Backbone=CLIP-L, Res.=512², Params (M)=304.20, FLOPS (G)=262.19↓15.6%, Peak Mem. (GB)=1.2163↓7.7%, Throughput (imgs/s)=41.56↑16.1%, fine-tuning=true2026.03 | 83.4 | |
| SemMAEModel=ViT-B, Modality=RGB, Masking=Semantic, PT Epoch=800, Input Size=224 × 224, Evaluation Protocol=Fine-tuning2026.03 | 83.3 | |
| AutoMAEModel=ViT-B, Modality=RGB, Masking=Semantic, PT Epoch=800, Input Size=224 × 224, Evaluation Protocol=Fine-tuning2026.03 | 83.3 | |
| MultiMAEModel=ViT-B, Modality=RGB/Dep./Sem., Masking=Random, PT Epoch=1600, PT Cost=∼5.2×, Input Size=224 × 224, Evaluation Protocol=Fine-tuning2026.03 | 83.3 | |
| BEITBackbone=ViT-B, Supervision=DALLE, Forward Ratio=100%, Reconstruct Ratio=40%, Epochs=8002022.11 | 83.2 | |
| MoCo v3Model=ViT-B, Modality=RGB, PT Epoch=300, Input Size=224 × 224, Evaluation Protocol=Fine-tuning2026.03 | 83.2 | |
| BEiTModel=ViT-B, Modality=RGB, Masking=Random, PT Epoch=800, PT Cost=∼7.0×, Input Size=224 × 224, Evaluation Protocol=Fine-tuning2026.03 | 83.2 | |
| MFMModel=ViT-B, Modality=RGB/Frequency, Masking=Random, PT Epoch=300, PT Cost=∼1.1×, Input Size=224 × 224, Evaluation Protocol=Fine-tuning2026.03 | 83.1 | |
| MAE (reproduced)Model=ViT-B, Modality=RGB, Masking=Random, PT Epoch=400, PT Cost=∼1.0×, Note=reproduced using official code, Input Size=224 × 224, Evaluation Protocol=Fine-tuning2026.03 | 82.9 | |
| UnMAEModel=ViT-B, Modality=RGB, Masking=Uniform, PT Epoch=200, Input Size=224 × 224, Evaluation Protocol=Fine-tuning2026.03 | 82.9 | |
| DINOModel=ViT-B, Modality=RGB, PT Epoch=300, Input Size=224 × 224, Evaluation Protocol=Fine-tuning2026.03 | 82.8 | |
| MultiMAE (reproduced)Model=ViT-B, Modality=RGB/Dep./Sem., Masking=Random, PT Epoch=400, PT Cost=∼1.3×, Note=reproduced using official code, Input Size=224 × 224, Evaluation Protocol=Fine-tuning2026.03 | 82.7 | |
| ScratchModel=ViT-B, Input Size=224 × 224, Evaluation Protocol=Fine-tuning2026.03 | 82.3 | |
| MonarchBackbone=CLIP-L, Res.=512², Params (M)=304.20, FLOPS (G)=269.44↓13.3%, Peak Mem. (GB)=1.2304↓6.7%, Throughput (imgs/s)=18.31↓48.9%, fine-tuning=true2026.03 | 81.5 | |
| NystromformerBackbone=CLIP-L, Res.=512², Params (M)=304.20, FLOPS (G)=265.20↓14.6%, Peak Mem. (GB)=1.2163↓7.7%, Throughput (imgs/s)=18.29↓50.8%, fine-tuning=true2026.03 | 80.7 | |
| CIMTarget=BEIT, Epochs PT=300, Backbone=ResNet-50, Fine-tuning=true2022.05 | 80.4 | |
| A2MIMTarget=RGB, Epochs PT=300, Backbone=ResNet-50, Fine-tuning=true2022.05 | 80.4 | |
| SwAVTarget=CL, Epochs PT=800, Backbone=ResNet-50, Fine-tuning=true2022.05 | 80.2 | |
| BYOLTarget=CL, Epochs PT=400, Backbone=ResNet-50, Fine-tuning=true2022.05 | 80.1 | |
| SimSiamTarget=CL, Epochs PT=400, Backbone=ResNet-50, Fine-tuning=true2022.05 | 80 | |
| SimCLRTarget=CL, Epochs PT=800, Backbone=ResNet-50, Fine-tuning=true2022.05 | 79.9 | |
| Barlow TwinsTarget=CL, Epochs PT=800, Backbone=ResNet-50, Fine-tuning=true2022.05 | 79.9 | |
| SimMIM+Target=RGB, Epochs PT=300, Backbone=ResNet-50, Fine-tuning=true, Reproduced=true2022.05 | 79.9 | |
| Sup.Target=Label, Epochs PT=90, Backbone=ResNet-50, Fine-tuning=true2022.05 | 79.8 | |
| MoCoV2Target=CL, Epochs PT=800, Backbone=ResNet-50, Fine-tuning=true2022.05 | 79.8 | |
| CosformerBackbone=CLIP-L, Res.=512², Params (M)=304.20, FLOPS (G)=265.41↓14.6%, Peak Mem. (GB)=1.2226↓7.2%, Throughput (imgs/s)=39.55↑8.2%, fine-tuning=true2026.03 | 70.6 | |
| PerformerBackbone=CLIP-L, Res.=512², Params (M)=304.20, FLOPS (G)=265.41↓14.6%, Peak Mem. (GB)=1.2164↓7.7%, Throughput (imgs/s)=36.48↑0.1%, fine-tuning=true2026.03 | 69.9 | |
| HedgehogBackbone=CLIP-L, Res.=512², Params (M)=305.77, FLOPS (G)=265.41↓14.5%, Peak Mem. (GB)=1.2221↓7.3%, Throughput (imgs/s)=37.44↑2.5%, fine-tuning=true2026.03 | 67.4 | |
| LoLCATSBackbone=CLIP-L, Res.=512², Params (M)=305.77, FLOPS (G)=265.41↓14.5%, Peak Mem. (GB)=1.2221↓7.3%, Throughput (imgs/s)=37.44↑2.5%, fine-tuning=true2026.03 | 67.21 | |
| LinformerBackbone=CLIP-L, Res.=512², Params (M)=305.77, FLOPS (G)=262.19↓15.6%, Peak Mem. (GB)=1.2221↓7.3%, Throughput (imgs/s)=45.41↑25.3%, fine-tuning=true2026.03 | 56.8 |