Multi-Task Adaptation on PASCAL-Context (test)
78.41Seg AccHyperformer
Evaluation results
| Method | Links | |||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| HyperformerBackbone=SwinTransformer-Base, Pre-trained=ImageNet-22k, Trainable Parameters (Encoder)=60.88M, Trainable Parameters (All)=63.92M2022.10 | 78.41 | 68.94 | 67.5 | 16.8 | 6.91 | — | — | — | — | — | — | |
| LoRABackbone=SwinTransformer-Base, Pre-trained=ImageNet-22k, Trainable Parameters (Encoder)=0.87M, Trainable Parameters (All)=4.31M2022.10 | 78.24 | 66.95 | 64.7 | 18.07 | 4.86 | — | — | — | — | — | — | |
| Polyhistor-LiteBackbone=SwinTransformer-Base, Pre-trained=ImageNet-22k, Down-projection ratio (p)=1, Trainable Parameters (Encoder)=1.29M, Trainable Parameters (All)=4.34M2022.10 | 77.91 | 68.02 | 66.89 | 16.54 | 8.08 | — | — | — | — | — | — | |
| Polyhistor-LiteBackbone=SwinTransformer-Base, Pre-trained=ImageNet-22k, Down-projection ratio (p)=32, Trainable Parameters (Encoder)=0.24M, Trainable Parameters (All)=3.28M2022.10 | 77.74 | 66.33 | 65.03 | 17.65 | 5.15 | — | — | — | — | — | — | |
| AdapterBackbone=SwinTransformer-Base, Pre-trained=ImageNet-22k, Trainable Parameters (Encoder)=3.64M, Trainable Parameters (All)=6.68M2022.10 | 77.22 | 65.95 | 63.8 | 18.38 | 3.35 | — | — | — | — | — | — | |
| BitfitBackbone=SwinTransformer-Base, Pre-trained=ImageNet-22k, Trainable Parameters (Encoder)=0.20M, Trainable Parameters (All)=3.24M2022.10 | 76.42 | 64.89 | 62.05 | 19.03 | 1.09 | — | — | — | — | — | — | |
| PHM layerBackbone=SwinTransformer-Base, Pre-trained=ImageNet-22k, Trainable Parameters (Encoder)=1.89M, Trainable Parameters (All)=4.94M2022.10 | 76.33 | 64.59 | 60.43 | 20.23 | -1.32 | — | — | — | — | — | — | |
| Compacter++Backbone=SwinTransformer-Base, Pre-trained=ImageNet-22k, Trainable Parameters (Encoder)=0.25M, Trainable Parameters (All)=3.29M2022.10 | 75.99 | 64.65 | 60.42 | 20.01 | -1.13 | — | — | — | — | — | — | |
| Low-rank adapterBackbone=SwinTransformer-Base, Pre-trained=ImageNet-22k, Trainable Parameters (Encoder)=0.34M, Trainable Parameters (All)=2.89M2022.10 | 75.65 | 64.75 | 60.5 | 20.03 | -1.21 | — | — | — | — | — | — | |
| VPT-deepBackbone=SwinTransformer-Base, Pre-trained=ImageNet-22k, Trainable Parameters (Encoder)=2.41M, Trainable Parameters (All)=5.45M2022.10 | 74.21 | 61.41 | 58.8 | 21.61 | -5.9 | — | — | — | — | — | — | |
| LoRABackbone=SwinTransformer-Base, Pre-training=ImageNet-1k, Trainable Parameters (Encoder)=0.87M, Trainable Parameters (All)=4.31M2022.10 | 74.1 | 61.57 | 63.87 | 18.55 | 2.63 | — | — | — | — | — | — | |
| Polyhistor-LiteBackbone=SwinTransformer-Base, Pre-training=ImageNet-1k, Down-projection ratio p=8, Trainable Parameters (Encoder)=0.29M, Trainable Parameters (All)=3.34M2022.10 | 73.92 | 62.15 | 65.37 | 17.7 | 4.53 | — | — | — | — | — | — | |
| Polyhistor-LiteBackbone=SwinTransformer-Base, Pre-training=ImageNet-1k, Down-projection ratio p=32, Trainable Parameters (Encoder)=0.24M, Trainable Parameters (All)=3.28M2022.10 | 73.8 | 61.32 | 64.64 | 17.92 | 3.57 | — | — | — | — | — | — | |
| Polyhistor-LiteBackbone=SwinTransformer-Base, Pre-training=ImageNet-1k, Down-projection ratio p=1, Trainable Parameters (Encoder)=1.29M, Trainable Parameters (All)=4.34M2022.10 | 73.7 | 63.32 | 66.5 | 16.93 | 6.38 | — | — | — | — | — | — | |
| Polyhistor-LiteBackbone=SwinTransformer-Base, Pre-training=ImageNet-1k, Down-projection ratio p=2, Trainable Parameters (Encoder)=0.62M, Trainable Parameters (All)=3.67M2022.10 | 73.69 | 63.04 | 66.56 | 17.3 | 5.8 | — | — | — | — | — | — | |
| HyperformerBackbone=SwinTransformer-Base, Pre-training=ImageNet-1k, Trainable Parameters (Encoder)=60.88M, Trainable Parameters (All)=63.92M2022.10 | 73.6 | 63.82 | 67.31 | 16.9 | 6.91 | — | — | — | — | — | — | |
| Polyhistor-LiteBackbone=SwinTransformer-Base, Pre-training=ImageNet-1k, Down-projection ratio p=4, Trainable Parameters (Encoder)=0.39M, Trainable Parameters (All)=3.43M2022.10 | 73.57 | 62.04 | 65.84 | 17.7 | 4.55 | — | — | — | — | — | — | |
| Fine-tuning DecodersBackbone=SwinTransformer-Base, Pre-trained=ImageNet-22k, Trainable Parameters (Encoder)=0.00M, Trainable Parameters (All)=3.04M2022.10 | 73.33 | 60.56 | 59.13 | 21.38 | -5.94 | — | — | — | — | — | — | |
| AdapterBackbone=SwinTransformer-Base, Pre-training=ImageNet-1k, Trainable Parameters (Encoder)=3.64M, Trainable Parameters (All)=6.68M2022.10 | 73.29 | 60.3 | 62.42 | 18.66 | 1.1 | — | — | — | — | — | — | |
| Relative biasBackbone=SwinTransformer-Base, Pre-trained=ImageNet-22k, Trainable Parameters (Encoder)=0.06M, Trainable Parameters (All)=3.11M2022.10 | 72.86 | 60.64 | 58.44 | 21.51 | -6.53 | — | — | — | — | — | — | |
| Low-rank adapterBackbone=SwinTransformer-Base, Pre-training=ImageNet-1k, Trainable Parameters (Encoder)=0.34M, Trainable Parameters (All)=2.89M2022.10 | 72.13 | 59.1 | 59.81 | 20.28 | -3.01 | — | — | — | — | — | — | |
| Compacter++Backbone=SwinTransformer-Base, Pre-training=ImageNet-1k, Trainable Parameters (Encoder)=0.25M, Trainable Parameters (All)=3.29M2022.10 | 72 | 59.11 | 59.73 | 20.41 | -3.25 | — | — | — | — | — | — | |
| AdapterBackbone=Pyramid Vision Transformer-Small, Num. of Trainable Parameters (Encoder)=0.79M, Num. of Trainable Parameters (All)=2.90M2022.10 | 71.94 | 56.38 | 64.16 | 18.75 | -1.97 | — | — | — | — | — | — | |
| BitfitBackbone=SwinTransformer-Base, Pre-training=ImageNet-1k, Trainable Parameters (Encoder)=0.20M, Trainable Parameters (All)=3.24M2022.10 | 71.93 | 59.12 | 60.67 | 20.08 | -2.46 | — | — | — | — | — | — | |
| PHM layerBackbone=SwinTransformer-Base, Pre-training=ImageNet-1k, Trainable Parameters (Encoder)=1.89M, Trainable Parameters (All)=4.94M2022.10 | 71.93 | 59.11 | 59.71 | 20.35 | -3.21 | — | — | — | — | — | — | |
| LoRABackbone=Pyramid Vision Transformer-Small, Num. of Trainable Parameters (Encoder)=0.30M, Num. of Trainable Parameters (All)=2.41M2022.10 | 71.89 | 56.9 | 64.27 | 18.48 | -1.35 | — | — | — | — | — | — | |
| HyperformerBackbone=SwinTransformer-Tiny, Number of Trainable Parameters (Encoder)=72.77, Number of Trainable Parameters (All)=75.322022.10 | 71.43 | 60.73 | 65.54 | 17.77 | 2.64 | — | — | — | — | — | — | |
| BitfitBackbone=Pyramid Vision Transformer-Small, Num. of Trainable Parameters (Encoder)=0.22M, Num. of Trainable Parameters (All)=2.34M2022.10 | 71.41 | 55.71 | 64.08 | 18.69 | -2.38 | — | — | — | — | — | — | |
| Polyhistor-LiteBackbone=Pyramid Vision Transformer-Small, p=1, Num. of Trainable Parameters (Encoder)=5.21M, Num. of Trainable Parameters (All)=7.32M2022.10 | 71 | 57.52 | 65.83 | 17.83 | 0.13 | — | — | — | — | — | — | |
| Polyhistor-LiteBackbone=Pyramid Vision Transformer-Small, p=32, Num. of Trainable Parameters (Encoder)=0.29M, Num. of Trainable Parameters (All)=2.40M2022.10 | 70.93 | 56.71 | 65 | 17.95 | -0.73 | — | — | — | — | — | — | |
| PolyhistorBackbone=SwinTransformer-Tiny, Number of Trainable Parameters (Encoder)=6.41, Number of Trainable Parameters (All)=8.962022.10 | 70.87 | 59.54 | 65.47 | 17.47 | 2.34 | — | — | — | — | — | — | |
| PHM layerBackbone=Pyramid Vision Transformer-Small, Num. of Trainable Parameters (Encoder)=0.42M, Num. of Trainable Parameters (All)=2.53M2022.10 | 70.81 | 55.02 | 63.51 | 18.75 | -3.2 | — | — | — | — | — | — | |
| HyperformerBackbone=Pyramid Vision Transformer-Small, Num. of Trainable Parameters (Encoder)=14.03M, Num. of Trainable Parameters (All)=16.14M2022.10 | 70.81 | 57.76 | 65.49 | 17.75 | 0.14 | — | — | — | — | — | — | |
| Low-rank adapterBackbone=Pyramid Vision Transformer-Small, Num. of Trainable Parameters (Encoder)=0.25M, Num. of Trainable Parameters (All)=2.36M2022.10 | 70.72 | 55.34 | 63.39 | 18.7 | -3.08 | — | — | — | — | — | — | |
| Single-task Full Fine-tuningBackbone=SwinTransformer-Base, Pre-trained=ImageNet-22k, Trainable Parameters (Encoder)=346.96M, Trainable Parameters (All)=350.01M2022.10 | 70.72 | 67.47 | 61 | 18.73 | 0 | — | — | — | — | — | — | |
| Compacter++Backbone=Pyramid Vision Transformer-Small, Num. of Trainable Parameters (Encoder)=0.09M, Num. of Trainable Parameters (All)=2.20M2022.10 | 70.29 | 54.8 | 63.16 | 18.82 | -3.71 | — | — | — | — | — | — | |
| Polyhistor-LiteBackbone=SwinTransformer-Tiny, Number of Trainable Parameters (Encoder)=0.41, Number of Trainable Parameters (All)=2.962022.10 | 70.24 | 59.12 | 64.75 | 17.4 | 1.74 | — | — | — | — | — | — | |
| Shared AdapterBackbone=SwinTransformer-Tiny, Number of Trainable Parameters (Encoder)=2.20, Number of Trainable Parameters (All)=4.742022.10 | 70.21 | 59.15 | 62.29 | 19.26 | -1.83 | — | — | — | — | — | — | |
| LORABackbone=SwinTransformer-Tiny, Number of Trainable Parameters (Encoder)=0.32, Number of Trainable Parameters (All)=2.872022.10 | 70.12 | 57.73 | 61.9 | 18.96 | -2.17 | — | — | — | — | — | — | |
| AdapterBackbone=SwinTransformer-Tiny, Number of Trainable Parameters (Encoder)=8.69, Number of Trainable Parameters (All)=11.242022.10 | 69.21 | 57.38 | 61.28 | 18.83 | -2.71 | — | — | — | — | — | — | |
| Fine-tuning DecodersBackbone=SwinTransformer-Base, Pre-training=ImageNet-1k, Trainable Parameters (Encoder)=0.00M, Trainable Parameters (All)=3.04M2022.10 | 68.98 | 55.57 | 58.37 | 21.36 | -7.55 | — | — | — | — | — | — | |
| Single-task Full Fine-tuningBackbone=Pyramid Vision Transformer-Small, Num. of Trainable Parameters (Encoder)=95.88M, Num. of Trainable Parameters (All)=97.99M2022.10 | 68.81 | 61.27 | 62.67 | 17.55 | 0 | — | — | — | — | — | — | |
| VPT-deepBackbone=SwinTransformer-Base, Pre-training=ImageNet-1k, Trainable Parameters (Encoder)=2.41M, Trainable Parameters (All)=5.45M2022.10 | 68.8 | 55.85 | 58.25 | 21.37 | -7.57 | — | — | — | — | — | — | |
| Multi-task Full Fine-tuningBackbone=SwinTransformer-Tiny, Number of Trainable Parameters (Encoder)=27.51, Number of Trainable Parameters (All)=30.062022.10 | 68.71 | 62.13 | 64.18 | 17.35 | 2.23 | — | — | — | — | — | — | |
| BitfitBackbone=SwinTransformer-Tiny, Number of Trainable Parameters (Encoder)=0.30, Number of Trainable Parameters (All)=2.852022.10 | 68.57 | 55.99 | 60.64 | 19.42 | -4.6 | — | — | — | — | — | — | |
| PHM layerBackbone=SwinTransformer-Tiny, Number of Trainable Parameters (Encoder)=0.59, Number of Trainable Parameters (All)=3.142022.10 | 68.55 | 56.28 | 60.35 | 19.23 | -4.34 | — | — | — | — | — | — | |
| Relative biasBackbone=SwinTransformer-Base, Pre-training=ImageNet-1k, Trainable Parameters (Encoder)=0.06M, Trainable Parameters (All)=3.11M2022.10 | 68.44 | 55.7 | 57.27 | 21.63 | -8.51 | — | — | — | — | — | — | |
| Low-rank adapterBackbone=SwinTransformer-Tiny, Number of Trainable Parameters (Encoder)=0.34, Number of Trainable Parameters (All)=2.892022.10 | 68.31 | 56.53 | 60.29 | 19.36 | -4.54 | — | — | — | — | — | — | |
| CompacterBackbone=SwinTransformer-Tiny, Number of Trainable Parameters (Encoder)=0.23, Number of Trainable Parameters (All)=2.782022.10 | 68.08 | 56.41 | 60.08 | 19.22 | -4.55 | — | — | — | — | — | — | |
| Single-task Full Fine-tuningBackbone=SwinTransformer-Base, Pre-training=ImageNet-1k, Trainable Parameters (Encoder)=346.96M, Trainable Parameters (All)=350.01M2022.10 | 67.88 | 64.47 | 61.26 | 18.85 | 0 | — | — | — | — | — | — | |
| PolyhistorBackbone=MoBY-Tiny [33], Number of Trainable Parameters (Encoder/All)=6.41/8.96, p (down-projection ratio)=16, task embedding size=642022.10 | 67.69 | 59.32 | 65.15 | 17.43 | 0.0205 | — | — | — | — | — | — | |
| Compacter++Backbone=SwinTransformer-Tiny, Number of Trainable Parameters (Encoder)=0.11, Number of Trainable Parameters (All)=2.662022.10 | 67.26 | 55.69 | 59.47 | 19.54 | -5.84 | — | — | — | — | — | — | |
| Polyhistor-LiteBackbone=MoBY-Tiny [33], Number of Trainable Parameters (Encoder/All)=0.41/2.96, p (down-projection ratio)=2, task embedding size=642022.10 | 67.23 | 58.9 | 64.62 | 17.72 | 0.0109 | — | — | — | — | — | — | |
| Single-task Full Fine-tuningBackbone=SwinTransformer-Tiny, Number of Trainable Parameters (Encoder)=110.07, Number of Trainable Parameters (All)=112.622022.10 | 67.21 | 61.93 | 62.35 | 17.97 | 0 | — | — | — | — | — | — | |
| HyperformerBackbone=MoBY-Tiny [33], Number of Trainable Parameters (Encoder/All)=19.29/44.252022.10 | 66.5 | 58.97 | 66.02 | 17.61 | 0.0156 | — | — | — | — | — | — | |
| LoRABackbone=MoBY-Tiny [33], Number of Trainable Parameters (Encoder/All)=0.32/2.872022.10 | 65.64 | 57.66 | 62.29 | 18.47 | -0.0199 | — | — | — | — | — | — | |
| Single-task Full Fine-tuningBackbone=MoBY-Tiny [33], Number of Trainable Parameters (Encoder/All)=110.07/112.622022.10 | 65.52 | 61.78 | 62.05 | 18.14 | 0 | — | — | — | — | — | — | |
| AdapterBackbone=MoBY-Tiny [33], Number of Trainable Parameters (Encoder/All)=8.69/11.242022.10 | 65 | 56.66 | 60.84 | 18.64 | -0.0345 | — | — | — | — | — | — | |
| Fine-tuning DecodersBackbone=Pyramid Vision Transformer-Small, Num. of Trainable Parameters (Encoder)=0.00M, Num. of Trainable Parameters (All)=2.11M2022.10 | 64.86 | 51.18 | 61.54 | 19.55 | -8.85 | — | — | — | — | — | — | |
| VPT-deepBackbone=SwinTransformer-Tiny, Number of Trainable Parameters (Encoder)=0.88, Number of Trainable Parameters (All)=3.432022.10 | 64.35 | 52.54 | 58.15 | 21.07 | -10.85 | — | — | — | — | — | — | |
| Relative biasBackbone=SwinTransformer-Tiny, Number of Trainable Parameters (Encoder)=0.09, Number of Trainable Parameters (All)=2.642022.10 | 63.51 | 52.35 | 57.74 | 21.07 | -11.4 | — | — | — | — | — | — | |
| BitfitBackbone=MoBY-Tiny [33], Number of Trainable Parameters (Encoder/All)=0.30/2.852022.10 | 63.43 | 54.9 | 59.5 | 19.8 | -0.069 | — | — | — | — | — | — | |
| Low-rank adapterBackbone=MoBY-Tiny [33], Number of Trainable Parameters (Encoder/All)=0.34/2.892022.10 | 63.3 | 55.24 | 59.72 | 19.14 | -0.0582 | — | — | — | — | — | — | |
| PHM layerBackbone=MoBY-Tiny [33], Number of Trainable Parameters (Encoder/All)=0.59/3.142022.10 | 63.21 | 54.99 | 59.7 | 19.13 | -0.0595 | — | — | — | — | — | — | |
| Fine-tuning DecodersBackbone=SwinTransformer-Tiny, Number of Trainable Parameters (Encoder)=0.00, Number of Trainable Parameters (All)=2.552022.10 | 63.14 | 52.37 | 58.39 | 20.89 | -11.02 | — | — | — | — | — | — | |
| VPT-shallowBackbone=SwinTransformer-Tiny, Number of Trainable Parameters (Encoder)=0.02, Number of Trainable Parameters (All)=2.572022.10 | 62.96 | 52.27 | 58.31 | 20.9 | -11.18 | — | — | — | — | — | — | |
| Compacter++Backbone=MoBY-Tiny [33], Number of Trainable Parameters (Encoder/All)=0.11/2.662022.10 | 62.31 | 54.69 | 59.43 | 19.58 | -0.0714 | — | — | — | — | — | — | |
| Fine-tuning DecodersBackbone=MoBY-Tiny [33], Number of Trainable Parameters (Encoder/All)=0.00/2.552022.10 | 59.64 | 52.97 | 59.6 | 19.88 | -0.0921 | — | — | — | — | — | — | |
| VPT-shallowBackbone=MoBY-Tiny [33], Number of Trainable Parameters (Encoder/All)=0.02/2.572022.10 | 59.5 | 52.84 | 59.48 | 19.88 | -0.0936 | — | — | — | — | — | — | |
| VPT-deepBackbone=MoBY-Tiny [33], Number of Trainable Parameters (Encoder/All)=0.88/3.432022.10 | 56.15 | 50.3 | 57.22 | 20.71 | -0.1372 | — | — | — | — | — | — | |
| AdaShareNumber of Parameters=1.002023.03 | — | — | — | — | — | -1.4 | 4 | -0.5 | -0.7 | 0 | 0.3 | |
| Cross-StitchNumber of Parameters=5.002023.03 | — | — | — | — | — | -1.3 | 3.6 | -0.2 | -1.4 | 0 | 0.1 | |
| Flow-based Reduction (FBR)M (flow constant)=5, Number of Parameters=1.932023.03 | — | — | — | — | — | -0.3 | 3.4 | 0.9 | 0 | 0 | 0.8 | |
| Flow-based Reduction (FBR)M (flow constant)=7, Number of Parameters=1.912023.03 | — | — | — | — | — | 0 | -0.2 | 1.7 | 1.4 | 0 | 0.6 | |
| Flow-based Reduction (FBR)M (flow constant)=9, Number of Parameters=2.312023.03 | — | — | — | — | — | 0 | 3.6 | 1.8 | 1.4 | 0 | 1.4 | |
| LTBNumber of Parameters=3.192023.03 | — | — | — | — | — | -6.9 | -1.9 | 0.2 | -1.4 | 0 | -2 | |
| MTANNumber of Parameters=5.212023.03 | — | — | — | — | — | -3.6 | -0.7 | -0.3 | -5 | -6 | -3.1 | |
| NDDR-CNNNumber of Parameters=5.612023.03 | — | — | — | — | — | -1.1 | -2.6 | 0 | -5 | 0 | -1.7 | |
| PHNNumber of Parameters=2.512023.03 | — | — | — | — | — | -6.6 | -1.6 | -1 | 0 | 0 | -1.8 | |
| Shared BottomNumber of Parameters=1.002023.03 | — | — | — | — | — | -6.6 | -0.7 | -3.4 | -14.3 | 0 | -5 | |
| Single-TaskNumber of Parameters=5.002023.03 | — | — | — | — | — | 0 | 0 | 0 | 0 | 0 | 0 | |
| SluiceNumber of Parameters=5.002023.03 | — | — | — | — | — | -1.6 | -1.2 | -0.5 | -2.9 | -6 | -2.4 |