Benchmarks

Task NameDataset NameSOTA ResultTrendResults

Deep Leakage Defense

CIFAR-10

1SSIM

Backdoor detection

CIFAR-10 imbalanced µ=0.9, ρ=100 (test)

100Badnets TPR

Backdoor Sample Detection

CIFAR-10 imbalanced mu=0.9, rho=10 (train test)

96.7Badnets TPR

Backdoor Sample Detection

CIFAR-10 balanced rho=1 (train test)

100Badnets TPR

Black-box DNN Watermarking

CIFAR-100

75.87CDA

Black-box DNN Watermarking

CIFAR-10

93.16CDA

Image Classification

CIFAR-10 standard (test)

88.07Accuracy

Random Data Forgetting

CIFAR-100 (test)

0.27Forgetting (Df)

Random Data Forgetting

CIFAR-10 (test)

98.4Df

Image Classification

CIFAR-10 FedAvg with 50 clients, beta=0.5 non-IID (test)

78.28Accuracy

Image Classification

CIFAR-10 Centralized original (test)

86.64Accuracy

Image Classification

CIFAR-100 (test)

49.75Accuracy (beta=0.1)

Backdoor Defense

CIFAR-100 (test)

79.7BadNet Clean Accuracy

Image Classification

CIFAR-10 (test)

96.7Accuracy (Symmetric 20%)

Image Denoising

CIFAR-10 (test)

25.91PSNR

Federated Image Classification

CIFAR10 (test)

74.07GA

OOD Detection

CIFAR10 Dfar

94.6AUROC

Multi-task Binary Classification

CIFAR-MTL (test)

59.97Accuracy

Image Classification

CIFAR-100 i.i.d. 500 clients 2% participation (test)

41.16Accuracy (500R)

Image Classification

CIFAR-100 Dirichlet 0.6 100 clients 5% participation (test)

58.82Accuracy (500R)

Image Classification

CIFAR-10 Dirichlet 0.6, 100 clients, 5% participation (test)

88.05Accuracy (500 Rounds)

Federated Learning

CIFAR-10 (test)

73.61Accuracy (500R)

Federated Learning

CIFAR-100 100 clients Dirichlet 0.3

49.56Accuracy (500 Rounds)

Federated Learning

CIFAR-10 100 clients Dirichlet 0.3

77.76Accuracy (Round 500)

Image Classification

CIFAR100 ISLVRC2012 (test)

86.5Acc (CIFAR100)