Diagonal Fisher Information Matrix Estimation on CIFAR-100
11RelMAEHutchinson's FIM estimator
Evaluation results
| Method | Links | ||
|---|---|---|---|
| Hutchinson's FIM estimatorBackbone=ResNet-502025.05 | 11 | 0.99 | |
| Diagonal Gauss-Newton FIM estimatorBackbone=ResNet-502025.05 | 11 | 1.01 | |
| Hutchinson's FIM estimatorBackbone=EfficientNet-B02025.05 | 11 | 1 | |
| Diagonal Gauss-Newton FIM estimatorBackbone=EfficientNet-B02025.05 | 12 | 1 | |
| Empirical FIMBackbone=ResNet-502025.05 | 17 | 1 | |
| Empirical FIMBackbone=EfficientNet-B02025.05 | 17 | 1 | |
| Low-rank(2) FIM estimatorBackbone=ResNet-502025.05 | 95 | 0.46 | |
| Low-rank(2) FIM estimatorBackbone=EfficientNet-B02025.05 | 96 | 0.5 | |
| Low-rank FIM estimatorBackbone=ResNet-502025.05 | 97 | 0.97 | |
| Low-rank FIM estimatorBackbone=EfficientNet-B02025.05 | 98 | 0.98 |