Image Classification on MNIST (Accuracy, F1, and Efficiency Metrics)
99.8AccuracyCentralized
Evaluation results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| CentralizedDevices=Server, Clients=5, Rounds=1002026.06 | 99.8 | — | — | — | — | |
| Bray-Curtis (Norm.)Backbone=ResNet502026.03 | 99.62 | 99.61 | 25.2889 | 0.8453 | 4 | |
| Cosine (Unst.)Backbone=ResNet502026.03 | 99.6 | 99.6 | 26.1851 | 0.6888 | 6 | |
| Fine-tuned (STL)Backbone=CLIP-ViT-B/32, Bit-width=Full-precision, Training=Single-task learning2026.05 | 99.6 | — | — | — | — | |
| Cosine (Stable)Backbone=ResNet502026.03 | 99.53 | 99.53 | 26.4064 | 0.6974 | 6 | |
| QSplitFLDevices=All, Clients=5, Rounds=1002026.06 | 99.47 | — | — | — | — | |
| Mahalanobis (Chol.)Backbone=ResNet502026.03 | 99.38 | 99.38 | 31.9246 | 0.9966 | 1 | |
| EuclideanBackbone=ResNet502026.03 | 99.34 | 99.34 | 24.457 | 0.9998 | 1 | |
| WUDI-MergingBackbone=CLIP-ViT-B/32, Merging Strategy=WUDI-Merging, Bit-width=Full-precision2026.05 | 99.3 | — | — | — | — | |
| WUDI-Merging w/ GPTQBackbone=CLIP-ViT-B/32, Merging Strategy=WUDI-Merging, Quantization Algorithm=GPTQ, Bit-width=4-bit2026.05 | 99.3 | — | — | — | — | |
| WUDI-Merging w/ E-PMQBackbone=CLIP-ViT-B/32, Merging Strategy=WUDI-Merging, Quantization Algorithm=E-PMQ, Bit-width=4-bit2026.05 | 99.3 | — | — | — | — | |
| SplitFedDevices=All, Clients=5, Rounds=1002026.06 | 99.2 | — | — | — | — | |
| WUDI-Merging w/ RTNBackbone=CLIP-ViT-B/32, Merging Strategy=WUDI-Merging, Quantization Algorithm=RTN, Bit-width=4-bit2026.05 | 99.1 | — | — | — | — | |
| WUDI-Merging w/ AWQBackbone=CLIP-ViT-B/32, Merging Strategy=WUDI-Merging, Quantization Algorithm=AWQ, Bit-width=4-bit2026.05 | 99.1 | — | — | — | — | |
| BaselineBackbone=ResNet502026.03 | 99.09 | 99.09 | 29.36 | 0.4204 | 50 | |
| Simple Averaging w/ E-PMQBackbone=CLIP-ViT-B/32, Merging Strategy=Simple Averaging, Quantization Algorithm=E-PMQ, Bit-width=4-bit2026.05 | 99 | — | — | — | — | |
| TIES-Merging w/ E-PMQBackbone=CLIP-ViT-B/32, Merging Strategy=TIES-Merging, Quantization Algorithm=E-PMQ, Bit-width=4-bit, Lambda=0.32026.05 | 99 | — | — | — | — | |
| Task Arithmetic w/ E-PMQBackbone=CLIP-ViT-B/32, Merging Strategy=Task Arithmetic, Quantization Algorithm=E-PMQ, Bit-width=4-bit, Lambda=0.32026.05 | 98.9 | — | — | — | — | |
| Bray-Curtis (Norm.)Backbone=CNN2026.03 | 98.89 | 98.88 | 1.1348 | 0.7225 | 13.6667 | |
| Mahalanobis (Chol.)Backbone=CNN2026.03 | 98.79 | 98.79 | 1.0639 | 0.4673 | 36.3333 | |
| Minkowski (p=3.0)Backbone=CNN2026.03 | 98.77 | 98.76 | 1.1154 | 0.4195 | 49.3333 | |
| Hamming (Soft)Backbone=CNN2026.03 | 98.33 | 98.32 | 1.1815 | 0.3089 | 50 | |
| EuclideanBackbone=CNN2026.03 | 98.31 | 98.31 | 1.1543 | 0.4413 | 20.3333 | |
| FedProxDevices=High-end, Clients=5, Rounds=1002026.06 | 98.1 | — | — | — | — | |
| BaselineBackbone=CNN2026.03 | 97.82 | 97.82 | 1.19 | 0.5856 | 10.6667 | |
| FedAvgDevices=High-end, Clients=5, Rounds=1002026.06 | 97.8 | — | — | — | — | |
| q-FedAvgDevices=High-end, Clients=5, Rounds=1002026.06 | 97.5 | — | — | — | — | |
| Task ArithmeticBackbone=CLIP-ViT-B/32, Merging Strategy=Task Arithmetic, Bit-width=Full-precision, Lambda=0.32026.05 | 96.1 | — | — | — | — | |
| Task Arithmetic w/ GPTQBackbone=CLIP-ViT-B/32, Merging Strategy=Task Arithmetic, Quantization Algorithm=GPTQ, Bit-width=4-bit, Lambda=0.32026.05 | 95.9 | — | — | — | — | |
| Task Arithmetic w/ AWQBackbone=CLIP-ViT-B/32, Merging Strategy=Task Arithmetic, Quantization Algorithm=AWQ, Bit-width=4-bit, Lambda=0.32026.05 | 95.3 | — | — | — | — | |
| Task Arithmetic w/ RTNBackbone=CLIP-ViT-B/32, Merging Strategy=Task Arithmetic, Quantization Algorithm=RTN, Bit-width=4-bit, Lambda=0.32026.05 | 95.1 | — | — | — | — | |
| TIES-MergingBackbone=CLIP-ViT-B/32, Merging Strategy=TIES-Merging, Bit-width=Full-precision, Lambda=0.32026.05 | 94.1 | — | — | — | — | |
| TIES-Merging w/ GPTQBackbone=CLIP-ViT-B/32, Merging Strategy=TIES-Merging, Quantization Algorithm=GPTQ, Bit-width=4-bit, Lambda=0.32026.05 | 93.9 | — | — | — | — | |
| TIES-Merging w/ AWQBackbone=CLIP-ViT-B/32, Merging Strategy=TIES-Merging, Quantization Algorithm=AWQ, Bit-width=4-bit, Lambda=0.32026.05 | 93.3 | — | — | — | — | |
| TIES-Merging w/ RTNBackbone=CLIP-ViT-B/32, Merging Strategy=TIES-Merging, Quantization Algorithm=RTN, Bit-width=4-bit, Lambda=0.32026.05 | 93.1 | — | — | — | — | |
| Simple Averaging w/ GPTQBackbone=CLIP-ViT-B/32, Merging Strategy=Simple Averaging, Quantization Algorithm=GPTQ, Bit-width=4-bit2026.05 | 86.4 | — | — | — | — | |
| Simple AveragingBackbone=CLIP-ViT-B/32, Merging Strategy=Simple Averaging, Bit-width=Full-precision2026.05 | 86.3 | — | — | — | — | |
| Simple Averaging w/ RTNBackbone=CLIP-ViT-B/32, Merging Strategy=Simple Averaging, Quantization Algorithm=RTN, Bit-width=4-bit2026.05 | 84.4 | — | — | — | — | |
| Simple Averaging w/ AWQBackbone=CLIP-ViT-B/32, Merging Strategy=Simple Averaging, Quantization Algorithm=AWQ, Bit-width=4-bit2026.05 | 84.3 | — | — | — | — | |
| Pre-trainedBackbone=CLIP-ViT-B/32, Bit-width=Full-precision2026.05 | 48.2 | — | — | — | — |