Image Classification on ImageNet (val) (Top-1, Params, Entropy)
79.9Top-1 AccBaseline
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| BaselineModel=DeiT-Small2023.09 | 79.9 | — | 16.7 | 19.1747 | |
| Q-ViTModel=DeiT-Small, Separate Codebook (Layer)=true, Separate Codebook (In-ch)=true, Separate Codebook (Attn)=true, w/o FL-Bias=false2023.09 | 78.1 | — | 11.3 | 3.0669 | |
| Q-ViTModel=DeiT-Small, Separate Codebook (Layer)=true, Separate Codebook (In-ch)=true, Separate Codebook (Attn)=true, w/o FL-Bias=true2023.09 | 78.1 | — | 10.4 | 0.2571 | |
| PWFNModel=DeiT-Small, Separate Codebook (Layer)=false, Separate Codebook (In-ch)=false, Separate Codebook (Attn)=false2023.09 | 78.1 | 78.5 | 2.7 | 0.0004 | |
| PWFN (no prior)Model=DeiT-Small, Separate Codebook (Layer)=false, Separate Codebook (In-ch)=false, Separate Codebook (Attn)=false2023.09 | 78 | 78.3 | 2.7 | 0.0004 | |
| LSQ+Model=DeiT-Small, Separate Codebook (Layer)=true, Separate Codebook (In-ch)=true, Separate Codebook (Attn)=false2023.09 | 77.8 | — | — | — | |
| BaselineModel=DenseNet1612023.09 | 77.8 | — | 17.1 | 26.4232 | |
| PWFNModel=DenseNet161, Separate Codebook (Layer)=false, Separate Codebook (In-ch)=false2023.09 | 77.6 | 78 | 1.1 | 0.0001 | |
| PWFNModel=ResNet-50, Separate Codebook (Layer)=false, Separate Codebook (In-ch)=false2023.09 | 77.5 | 78.3 | 3.4 | 0.0003 | |
| PWFN (no prior)Model=ResNet-50, Separate Codebook (Layer)=false, Separate Codebook (In-ch)=false2023.09 | 77.2 | 78.1 | 3.5 | 0.0003 | |
| BaselineModel=ResNet-502023.09 | 76.1 | — | 24.2 | 19.9157 | |
| WFNModel=ResNet-50, Separate Codebook (Layer)=false, Separate Codebook (In-ch)=false2023.09 | 76 | — | 4.1 | 0.0003 | |
| LSQModel=ResNet-50, Separate Codebook (Layer)=true, Separate Codebook (In-ch)=false2023.09 | 75.8 | — | — | — | |
| PWFNModel=ResNet-34, Separate Codebook (Layer)=false, Separate Codebook (In-ch)=false2023.09 | 74.3 | 74.6 | 1.8 | 0.0002 | |
| PWFN (no prior)Model=ResNet-34, Separate Codebook (Layer)=false, Separate Codebook (In-ch)=false2023.09 | 73.5 | 74.4 | 1.2 | 0.0001 | |
| APoTModel=ResNet-34, Separate Codebook (Layer)=true, Separate Codebook (In-ch)=false2023.09 | 73.4 | — | 6.8 | 0.0167 | |
| BaselineModel=ResNet-342023.09 | 73.3 | — | 24.1 | 19.0143 | |
| WFNModel=ResNet-34, Separate Codebook (Layer)=false, Separate Codebook (In-ch)=false2023.09 | 73 | — | 3.8 | 0.0002 | |
| BaselineModel=DeiT-Tiny2023.09 | 72.9 | — | 15.5 | 5.4811 | |
| LSQModel=ResNet-34, Separate Codebook (Layer)=true, Separate Codebook (In-ch)=false2023.09 | 71.9 | — | — | — | |
| PWFN (no prior)Model=DeiT-Tiny, Separate Codebook (Layer)=false, Separate Codebook (In-ch)=false, Separate Codebook (Attn)=false2023.09 | 71.4 | 71.6 | 2.8 | 0.0003 | |
| PWFNModel=DeiT-Tiny, Separate Codebook (Layer)=false, Separate Codebook (In-ch)=false, Separate Codebook (Attn)=false2023.09 | 71.2 | 71.5 | 2.8 | 0.0003 | |
| PWFNModel=ResNet-18, Separate Codebook (Layer)=false, Separate Codebook (In-ch)=false2023.09 | 70 | 70.1 | 2.5 | 0.0002 | |
| APoTModel=ResNet-18, Separate Codebook (Layer)=true, Separate Codebook (In-ch)=false2023.09 | 69.9 | — | 5.7 | 0.0014 | |
| WFNModel=ResNet-18, Separate Codebook (Layer)=false, Separate Codebook (In-ch)=false2023.09 | 69.7 | — | 3 | 0.0002 | |
| Q-ViTModel=DeiT-Tiny, Separate Codebook (Layer)=true, Separate Codebook (In-ch)=true, Separate Codebook (Attn)=true, w/o FL-Bias=false2023.09 | 69.6 | — | 11.5 | 1.1176 | |
| Q-ViTModel=DeiT-Tiny, Separate Codebook (Layer)=true, Separate Codebook (In-ch)=true, Separate Codebook (Attn)=true, w/o FL-Bias=true2023.09 | 69.6 | — | 10.5 | 0.1288 | |
| PWFN (no prior)Model=ResNet-18, Separate Codebook (Layer)=false, Separate Codebook (In-ch)=false2023.09 | 69.3 | 69.6 | 1.7 | 0.0001 | |
| BaselineModel=ResNet-182023.09 | 68.9 | — | 23.3 | 10.756 | |
| LSQModel=ResNet-18, Separate Codebook (Layer)=true, Separate Codebook (In-ch)=false2023.09 | 68.2 | — | — | — | |
| LSQ+Model=DeiT-Tiny, Separate Codebook (Layer)=true, Separate Codebook (In-ch)=true, Separate Codebook (Attn)=false2023.09 | 68.1 | — | — | — |