Multi-task Image Classification on 8-task vision suite
94.3Average PerformanceTALL-Masks
Evaluation Results
| Method | Links | |
|---|---|---|
| TALL-MasksBackbone=ViT-L/14, Task Index=Required, Params.=2.2×2025.03 | 94.3 | |
| IndividualBackbone=ViT-L/14, Task Index=-, Params.=8×2025.03 | 94.2 | |
| TSV-CBackbone=ViT-L/14, Task Index=Required, Params.=2.5×2025.03 | 93.8 | |
| TSV-M+AdaRankBackbone=ViT-L/14, Task Index=-, Params.=1×2025.03 | 93.7 | |
| CART+AdaRankBackbone=ViT-L/14, Task Index=-, Params.=1×2025.03 | 93.5 | |
| EMR-MergingBackbone=ViT-L/14, Task Index=Required, Params.=4.4×2025.03 | 93.5 | |
| TA+AdaRankBackbone=ViT-L/14, Task Index=-, Params.=1×2025.03 | 93 | |
| ReTeX (Sample-wise)Backbone=CLIP ViT-B/32, Task-ID=Unknown, Inference mode=Sample-wise, Inference cost (per input)=0.05s, VRAM (GB)=2.52026.06 | 92.2 | |
| ReTeX (Group: B = 64)Backbone=CLIP ViT-B/32, Task-ID=Unknown, Inference mode=Group, Batch size (B)=64, Inference cost (per input)=0.0015s, VRAM (GB)=2.72026.06 | 92.2 | |
| ReTeX-E (Sample-wise)Backbone=CLIP ViT-B/32, Task-ID=Unknown, Inference mode=Sample-wise, Inference cost (per input)=0.07s, VRAM (GB)=1.82026.06 | 92 | |
| ReTeX-E (Group: B = 64)Backbone=CLIP ViT-B/32, Task-ID=Unknown, Inference mode=Group, Batch size (B)=64, Inference cost (per input)=0.0015s, VRAM (GB)=2.02026.06 | 92 | |
| TALL-MasksBackbone=ViT-B/32, Task Index=Required, Params.=2.2×2025.03 | 90.8 | |
| IndividualBackbone=ViT-B/32, Task Index=-, Params.=8×2025.03 | 90.5 | |
| ID-EMR-MergingBackbone=CLIP ViT-B/32, Task-ID=Unknown, Model Merging type=Component selection, Inference cost (per input)=0.004s, VRAM (GB)=1.92026.06 | 90.5 | |
| WEMoEBackbone=CLIP ViT-B/32, Task-ID=Unknown, Model Merging type=Dynamic, Inference cost (per input)=0.02s, VRAM (GB)=4.32026.06 | 90.4 | |
| ID-LoRABackbone=CLIP ViT-B/32, Task-ID=Unknown, Model Merging type=Component selection, Inference cost (per input)=0.0009s, VRAM (GB)=1.92026.06 | 89.3 | |
| CART+AdaRankBackbone=ViT-B/32, Task Index=-, Params.=1×2025.03 | 89.2 | |
| TSV-CBackbone=ViT-B/32, Task Index=Required, Params.=2.5×2025.03 | 89 | |
| DaWinBackbone=CLIP ViT-B/32, Task-ID=Unknown, Model Merging type=Dynamic, Inference cost (per input)=0.63s, VRAM (GB)=5.52026.06 | 89 | |
| TSV-M+AdaRankBackbone=ViT-B/32, Task Index=-, Params.=1×2025.03 | 88.9 | |
| EMR-MergingBackbone=ViT-B/32, Task Index=Required, Params.=4.4×2025.03 | 88.8 | |
| MoW-MergingBackbone=CLIP ViT-B/32, Task-ID=Unknown, Model Merging type=Dynamic, Inference cost (per input)=0.008s, VRAM (GB)=4.92026.06 | 88.1 | |
| TA+AdaRankBackbone=ViT-B/32, Task Index=-, Params.=1×2025.03 | 87.9 | |
| Twin-MergingBackbone=CLIP ViT-B/32, Task-ID=Unknown, Model Merging type=Dynamic, Inference cost (per input)=0.03s, VRAM (GB)=3.22026.06 | 84 | |
| TIES-MergingBackbone=CLIP ViT-B/32, Task-ID=Unknown, Model Merging type=Static, Inference cost (per input)=0.0004s, VRAM (GB)=1.32026.06 | 75.1 | |
| Consensus TABackbone=CLIP ViT-B/32, Task-ID=Unknown, Model Merging type=Static, Inference cost (per input)=0.0004s, VRAM (GB)=1.32026.06 | 75 | |
| Task ArithmeticBackbone=CLIP ViT-B/32, Task-ID=Unknown, Model Merging type=Static, Inference cost (per input)=0.0004s, VRAM (GB)=1.32026.06 | 70.8 | |
| Weight AveragingBackbone=CLIP ViT-B/32, Task-ID=Unknown, Model Merging type=Static, Inference cost (per input)=0.0004s, VRAM (GB)=1.32026.06 | 66.3 |