Image Classification on Places-LT (test)
53.1Accuracy (Medium)LIFT w/ TTE
Evaluation results
| Method | Links | |||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| LIFT w/ TTEBackbone=ViT-B/16, Learnable Params.=0.18M, #Epochs=10, Training Protocol=Fine-tuning foundation model2023.09 | 53.1 | 52.2 | — | — | — | — | — | 51.7 | 50.9 | — | — | — | — | |
| Category Extrapolation (Ours)Pre-training paradigm=DINOv2, Data configuration=proposed method2024.10 | 52.4 | 50.8 | — | 49.4 | 49.2 | — | — | — | — | — | — | — | — | |
| LPTBackbone=ViT-B/16, Learnable Params.=1.01M, #Epochs=40+40, Training Protocol=Fine-tuning foundation model2023.09 | 52.3 | 50.1 | — | — | — | — | — | 49.3 | 46.9 | — | — | — | — | |
| LIFTBackbone=ViT-B/16, Learnable Params.=0.18M, #Epochs=10, Training Protocol=Fine-tuning foundation model2023.09 | 52.2 | 51.5 | — | — | — | — | — | 51.3 | 50.5 | — | — | — | — | |
| Memory Attention ModuleRetrieval=true, Backbone=ViT-B16, Visual Encoder=Fine-tuned2023.04 | 52 | — | — | 52.4 | 48.5 | 51.4 | — | — | — | — | — | — | — | |
| Baseline + SDPre-training paradigm=DINOv2, Data configuration=selected neighbor categories2024.10 | 51.6 | 49.9 | — | 49.3 | 47.3 | — | — | — | — | — | — | — | — | |
| BaselinePre-training paradigm=DINOv2, Data configuration=standard2024.10 | 51.3 | 49.5 | — | 49.2 | 46.1 | — | — | — | — | — | — | — | — | |
| Category Extrapolation (Ours)Pre-training paradigm=CLIP, Data configuration=proposed method2024.10 | 51 | 50.5 | — | 50 | 50.2 | — | — | — | — | — | — | — | — | |
| BALLADBackbone=ViT-B/16, Learnable Params.=149.62M, #Epochs=50+10, Training Protocol=Fine-tuning foundation model2023.09 | 50.2 | 49.5 | — | — | — | — | — | 49.3 | 48.4 | — | — | — | — | |
| Memory Attention ModuleRetrieval=true, Backbone=ViT-B16, Visual Encoder=Frozen2023.04 | 49.9 | — | — | 50.9 | 47.5 | 49.9 | — | — | — | — | — | — | — | |
| RAC†Retrieval=true, Backbone=ViT-B16, Visual Encoder=Fine-tuned2023.04 | 49.8 | — | — | 51.9 | 46.8 | 50 | — | — | — | — | — | — | — | |
| Baseline + SDPre-training paradigm=CLIP, Data configuration=selected neighbor categories2024.10 | 49.5 | 49.2 | — | 48.7 | 49.4 | — | — | — | — | — | — | — | — | |
| BaselinePre-training paradigm=CLIP, Data configuration=standard2024.10 | 48.6 | 48.4 | — | 47.9 | 48.9 | — | — | — | — | — | — | — | — | |
| VL-LTRBackbone=ViT-Base, Initialization=CLIP weights2021.11 | 48.5 | 50.1 | — | 54.2 | 42 | — | — | — | — | — | — | — | — | |
| VL-LTRRetrieval=false, Backbone=ViT-B16, Visual Encoder=Fine-tuned2023.04 | 48.5 | — | — | 54.2 | 42 | 50.1 | — | — | — | — | — | — | — | |
| VL-LTRBackbone=ViT-B/16, Learnable Params.=149.62M, #Epochs=100, Training Protocol=Fine-tuning with extra data2023.09 | 48.5 | 50.1 | — | — | — | — | — | 54.2 | 42 | — | — | — | — | |
| RAC2022.02 | 48.31 | 47.17 | — | 48.69 | 41.76 | — | — | — | — | — | — | — | — | |
| RACRetrieval=true, Backbone=ViT-B16, Visual Encoder=Fine-tuned2023.04 | 48.3 | — | — | 48.7 | 41.8 | 47.2 | — | — | — | — | — | — | — | |
| RAC†Retrieval=true, Backbone=ViT-B16, Visual Encoder=Frozen2023.04 | 48.3 | — | — | 50.3 | 42.5 | 47.9 | — | — | — | — | — | — | — | |
| RACBackbone=ViT-B/16, Learnable Params.=85.80M, #Epochs=30, Training Protocol=Fine-tuning with extra data2023.09 | 48.3 | 47.2 | — | — | — | — | — | 48.7 | 41.8 | — | — | — | — | |
| PaCo2022.02 | 47.9 | 41.2 | — | 36.1 | 35.3 | — | — | — | — | — | — | — | — | |
| PaCoBackbone=ResNet-1522021.11 | 47.9 | 41.2 | — | 36.1 | 35.3 | — | — | — | — | — | — | — | — | |
| PaCoRetrieval=false, Backbone=ResNext-101, Visual Encoder=Fine-tuned2023.04 | 47.9 | — | — | 36.1 | 35.3 | 41.2 | — | — | — | — | — | — | — | |
| PaCoBackbone=ResNet-152, Pre-training=ImageNet, RandAugment=true2022.09 | 47.9 | 41.2 | — | 36.1 | 35.3 | — | — | — | — | — | — | — | — | |
| PaCoBackbone=ResNet-152, Learnable Params.=58.14M, #Epochs=30, Training Protocol=Training from scratch (with ImageNet-1K pre-trained backbone)2023.09 | 47.9 | 41.2 | — | — | — | — | — | 36.1 | 35.3 | — | — | — | — | |
| PaCoBackbone=ResNet-152, Pre-trained=ImageNet2022.03 | 47.2 | — | — | 37.5 | 33.9 | 41.2 | — | — | — | — | — | — | — | |
| VL-LTRBackbone=ResNet-50, Initialization=CLIP weights2021.11 | 47.2 | 48 | — | 51.9 | 38.4 | — | — | — | — | — | — | — | — | |
| PaCo2022.09 | 47.2 | — | — | 37.5 | 33.9 | 41.2 | — | — | — | — | — | — | — | |
| PaCoBackbone=ResNet-1522023.02 | 47.2 | 41.2 | — | 37.5 | 33.9 | — | — | — | — | — | — | — | — | |
| PaCoBackbone=ResNet-152, Pre-training=ImageNet, RandAugment=false2022.09 | 47.2 | 41.2 | — | 37.5 | 33.9 | — | — | — | — | — | — | — | — | |
| GPaCoBackbone=ResNet-152, Pre-training=ImageNet, RandAugment=false2022.09 | 47.2 | 41.7 | — | 39.5 | 33 | — | — | — | — | — | — | — | — | |
| MLP ClassifierRetrieval=false, Backbone=ViT-B16, Visual Encoder=Frozen2023.04 | 46.1 | — | — | 48.6 | 41.3 | 46 | — | — | — | — | — | — | — | |
| Mean k-NNRetrieval=true, Backbone=ViT-B16, Visual Encoder=Frozen2023.04 | 45.2 | — | — | 44.3 | 45.5 | 44.9 | — | — | — | — | — | — | — | |
| Baseline + RDPre-training paradigm=DINOv2, Data configuration=random auxiliary data2024.10 | 45.2 | 45.2 | — | 47.2 | 41.3 | — | — | — | — | — | — | — | — | |
| Category Extrapolation (Ours)Pre-training paradigm=Scratch, Data configuration=proposed method2024.10 | 44.8 | 43.8 | — | 43.7 | 41.6 | — | — | — | — | — | — | — | — | |
| 3LSSLBackbone=ResNet-1522023.02 | 44.5 | 42 | — | 42.6 | 34.3 | — | — | — | — | — | — | — | — | |
| ResLTBackbone=ResNet-1522023.02 | 44.4 | 41 | — | 40.3 | 34.7 | — | — | — | — | — | — | — | — | |
| Linear ClassifierRetrieval=false, Backbone=ViT-B16, Visual Encoder=Frozen2023.04 | 44.4 | — | — | 44.5 | 44 | 44.3 | — | — | — | — | — | — | — | |
| Ours + cRTRepresentation Learning=Difficulty-Net2022.09 | 43.8 | — | — | 43 | 35 | 41.7 | — | — | — | — | — | — | — | |
| Ours + LWSRepresentation Learning=Difficulty-Net2022.09 | 43.7 | — | — | 41.4 | 36.9 | 41.5 | — | — | — | — | — | — | — | |
| Ours + LASRepresentation Learning=Difficulty-Net2022.09 | 43.7 | — | — | 42.4 | 36.6 | 41.7 | — | — | — | — | — | — | — | |
| ResLTBackbone=ResNet-152, Pre-trained=ImageNet2022.03 | 43.6 | — | — | 39.8 | 31.4 | 39.8 | — | — | — | — | — | — | — | |
| ResLTBackbone=ResNet-1522021.11 | 43.6 | 39.8 | — | 39.8 | 31.4 | — | — | — | — | — | — | — | — | |
| ResLTBackbone=ResNet-152, Pre-training=ImageNet, RandAugment=false2022.09 | 43.6 | 39.8 | — | 39.8 | 31.4 | — | — | — | — | — | — | — | — | |
| ResComBackbone=ResNet-152, Pre-trained=ImageNet2022.03 | 43.4 | — | — | 43 | 35.3 | 41.7 | — | — | — | — | — | — | — | |
| Baseline + RDPre-training paradigm=CLIP, Data configuration=random auxiliary data2024.10 | 43.4 | 43.3 | — | 45.1 | 40.2 | — | — | — | — | — | — | — | — | |
| MiSLASBackbone=ResNet-152, Pre-training=ImageNet pre-trained2021.04 | 43.3 | 40.4 | — | 39.6 | 36.1 | — | — | — | — | — | — | — | — | |
| MISLASBackbone=ResNet-152, Pre-trained=ImageNet2022.03 | 43.3 | — | — | 39.6 | 36.1 | 40.4 | — | — | — | — | — | — | — | |
| MISLAS2022.09 | 43.3 | — | — | 39.6 | 36.1 | 40.4 | — | — | — | — | — | — | — | |
| MISLASBackbone=ResNet-1522023.02 | 43.3 | 40.4 | — | 39.6 | 36.1 | — | — | — | — | — | — | — | — | |
| MISLASBackbone=ResNet-152, Pre-training=ImageNet, RandAugment=false2022.09 | 43.3 | 40.4 | — | 39.6 | 36.1 | — | — | — | — | — | — | — | — | |
| MiSLASBackbone=ResNet-152, Learnable Params.=58.14M, #Epochs=90+10, Training Protocol=Training from scratch (with ImageNet-1K pre-trained backbone)2023.09 | 43.3 | 40.4 | — | — | — | — | — | 39.6 | 36.1 | — | — | — | — | |
| TADEBackbone=ResNet-1522021.11 | 43.2 | 40.9 | — | 40.4 | 36.8 | — | — | — | — | — | — | — | — | |
| DisAlign2022.02 | 42.4 | 39.3 | — | 40.4 | 30.1 | — | — | — | — | — | — | — | — | |
| TADE2022.02 | 42.4 | 40.9 | — | 43.1 | 33.2 | — | — | — | — | — | — | — | — | |
| DisAlignBackbone=ResNet-152, Pre-trained=ImageNet2022.03 | 42.4 | — | — | 40.4 | 30.1 | 39.3 | — | — | — | — | — | — | — | |
| DisAlign2022.09 | 42.4 | — | — | 40.4 | 30.1 | 39.3 | — | — | — | — | — | — | — | |
| DisAlignBackbone=ResNet-152, Learnable Params.=58.14M, #Epochs=30, Training Protocol=Training from scratch (with ImageNet-1K pre-trained backbone)2023.09 | 42.4 | 39.3 | — | — | — | — | — | 40.4 | 30.1 | — | — | — | — | |
| Baseline + SDPre-training paradigm=Scratch, Data configuration=selected neighbor categories2024.10 | 42.1 | 41.6 | — | 43.4 | 36.9 | — | — | — | — | — | — | — | — | |
| RSGBackbone=ResNet-152, Pre-trained=ImageNet2022.03 | 41.4 | — | — | 41.9 | 32 | 39.3 | — | — | — | — | — | — | — | |
| LWS+mixupBackbone=ResNet-152, Pre-training=ImageNet pre-trained2021.04 | 41.3 | 39.7 | — | 41.7 | 33.1 | — | — | — | — | — | — | — | — | |
| GistNetBackbone=ResNet-152, Pre-trained=ImageNet2022.03 | 40.8 | — | — | 42.5 | 32.1 | 39.6 | — | — | — | — | — | — | — | |
| GistNetBackbone=ResNet-1522023.02 | 40.8 | 39.6 | — | 42.5 | 32.1 | — | — | — | — | — | — | — | — | |
| Decouple-τ-normBackbone=ResNet-152, Pre-trained=ImageNet2020.12 | 40.7 | 37.9 | — | 37.8 | 31.8 | — | — | — | — | — | — | — | — | |
| τ-normalizedBackbone=ResNet-152, Pre-training=ImageNet pre-trained2021.04 | 40.7 | 37.9 | — | 37.8 | 31.8 | — | — | — | — | — | — | — | — | |
| T-normBackbone=ResNet-152, Pre-trained=ImageNet2022.03 | 40.7 | — | — | 37.8 | 31.8 | 37.9 | — | — | — | — | — | — | — | |
| τ-normalizedBackbone=ResNet-1522021.11 | 40.7 | 37.9 | — | 37.8 | 31.8 | — | — | — | — | — | — | — | — | |
| Decouple-r-normBackbone=ResNet-152, Pre-training=ImageNet, RandAugment=false2022.09 | 40.7 | 37.9 | — | 37.8 | 31.8 | — | — | — | — | — | — | — | — | |
| LiVTBackbone=ViT-B/16, Learnable Params.=85.80M, #Epochs=100, Training Protocol=Training from scratch (with ImageNet-1K pre-trained backbone)2023.09 | 40.6 | 40.8 | — | — | — | — | — | 48.1 | 27.5 | — | — | — | — | |
| BaselinePre-training paradigm=Scratch, Data configuration=standard2024.10 | 40.5 | 39.9 | — | 43 | 33.3 | — | — | — | — | — | — | — | — | |
| ALA2022.02 | 40.1 | 40.1 | — | 43.9 | 32.9 | — | — | — | — | — | — | — | — | |
| ALABackbone=ResNet-152, Learnable Params.=58.14M, #Epochs=30, Training Protocol=Training from scratch (with ImageNet-1K pre-trained backbone)2023.09 | 40.1 | 40.1 | — | — | — | — | — | 43.9 | 32.9 | — | — | — | — | |
| BALMSTraining Strategy=Decoupled training, Backbone=ResNet-152, Pre-trained=ImageNet2020.07 | 39.8 | 38.7 | — | 41.2 | 31.6 | — | — | — | — | — | — | — | — | |
| BALMS2022.09 | 39.8 | — | — | 41.2 | 31.6 | 38.7 | — | — | — | — | — | — | — | |
| OLTR+LFMEBackbone=ResNet-152, Pre-training=ImageNet pre-trained2021.04 | 39.6 | 36.2 | — | 39.3 | 24.2 | — | — | — | — | — | — | — | — | |
| Balanced SoftmaxBackbone=ResNet-152, Pre-trained=ImageNet2020.12 | 39.3 | 38.6 | — | 42 | 30.5 | — | — | — | — | — | — | — | — | |
| Balanced SoftmaxBackbone=ResNet-152, Pre-trained=ImageNet2022.03 | 39.3 | — | — | 42 | 30.5 | 38.6 | — | — | — | — | — | — | — | |
| Balanced SoftmaxBackbone=ResNet-152, Pre-training=ImageNet, RandAugment=false2022.09 | 39.3 | 38.6 | — | 42 | 30.5 | — | — | — | — | — | — | — | — | |
| IEM2022.09 | 39.2 | — | — | 46.8 | 28 | 39.7 | — | — | — | — | — | — | — | |
| LWSTraining Strategy=Decoupled training, Backbone=ResNet-152, Pre-trained=ImageNet2020.07 | 39.1 | 37.6 | — | 40.6 | 28.6 | — | — | — | — | — | — | — | — | |
| Decouple-LWSBackbone=ResNet-152, Pre-trained=ImageNet2020.12 | 39.1 | 37.6 | — | 40.6 | 28.6 | — | — | — | — | — | — | — | — | |
| PC SoftmaxBackbone=ResNet-152, Pre-trained=ImageNet2020.12 | 39.1 | 38.7 | — | 43 | 29.6 | — | — | — | — | — | — | — | — | |
| LWSBackbone=ResNet-152, Pre-training=ImageNet pre-trained2021.04 | 39.1 | 37.6 | — | 40.6 | 28.6 | — | — | — | — | — | — | — | — | |
| Decouple-LWS2022.02 | 39.1 | 37.6 | — | 40.6 | 28.6 | — | — | — | — | — | — | — | — | |
| LWSBackbone=ResNet-152, Pre-trained=ImageNet2022.03 | 39.1 | — | — | 40.6 | 28.6 | 37.6 | — | — | — | — | — | — | — | |
| LWSBackbone=ResNet-1522021.11 | 39.1 | 37.6 | — | 40.6 | 28.6 | — | — | — | — | — | — | — | — | |
| LWS2022.09 | 39.1 | — | — | 40.6 | 28.6 | 37.6 | — | — | — | — | — | — | — | |
| LWSBackbone=ResNet-152, Learnable Params.=58.14M, #Epochs=90+10, Training Protocol=Training from scratch (with ImageNet-1K pre-trained backbone)2023.09 | 39.1 | 37.6 | — | — | — | — | — | 40.6 | 28.6 | — | — | — | — | |
| LADEBackbone=ResNet-152, Pre-trained=ImageNet2020.12 | 39 | 38.8 | — | 42.8 | 31.2 | — | — | — | — | — | — | — | — | |
| LADE2022.02 | 39 | 38.8 | — | 42.8 | 31.2 | — | — | — | — | — | — | — | — | |
| LADEBackbone=ResNet-152, Pre-trained=ImageNet2022.03 | 39 | — | — | 42.8 | 31.2 | 38.8 | — | — | — | — | — | — | — | |
| LADE2022.09 | 39 | — | — | 42.8 | 31.2 | 38.8 | — | — | — | — | — | — | — | |
| BaselineBackbone=ResNet-50, Initialization=CLIP weights2021.11 | 38.6 | 39.7 | — | 50.8 | 22.7 | — | — | — | — | — | — | — | — | |
| cRT+mixupBackbone=ResNet-152, Pre-training=ImageNet pre-trained2021.04 | 38.5 | 38.1 | — | 44.1 | 27.1 | — | — | — | — | — | — | — | — | |
| cRTTraining Strategy=Decoupled training, Backbone=ResNet-152, Pre-trained=ImageNet2020.07 | 37.6 | 36.7 | — | 42 | 24.9 | — | — | — | — | — | — | — | — | |
| cRTBackbone=ResNet-152, Pre-training=ImageNet pre-trained2021.04 | 37.6 | 36.7 | — | 42 | 24.9 | — | — | — | — | — | — | — | — | |
| CRTBackbone=ResNet-1522021.11 | 37.6 | 36.7 | — | 42 | 24.9 | — | — | — | — | — | — | — | — | |
| cRT2022.09 | 37.6 | — | — | 42 | 24.9 | 36.7 | — | — | — | — | — | — | — | |
| cRTBackbone=ResNet-152, Learnable Params.=58.14M, #Epochs=90+10, Training Protocol=Training from scratch (with ImageNet-1K pre-trained backbone)2023.09 | 37.6 | 36.7 | — | — | — | — | — | 42 | 24.9 | — | — | — | — | |
| Zero-Shot CLIPBackbone=ResNet-50, Initialization=CLIP weights2021.11 | 37.5 | 38 | — | 37.5 | 40.1 | — | — | — | — | — | — | — | — |