# Image Classification on FEMNIST (test) (Percentile Accuracy Analysis) Source: [Image Classification on FEMNIST (test) (Percentile Accuracy Analysis)]() Collection index: [llms.txt]() Data updated: 2026-05-21T13:07:26.203Z Evaluation of Image Classification performance on the test split of the FEMNIST dataset, focusing on mean accuracy and performance variance measured across the best and worst 10% of samples. A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. | Scope | Recorded value | | --- | --- | | Task | Image Classification | | Dataset (including any stated split) | FEMNIST (test) | | Metric | Mean Accuracy | | Unit | PERCENT | | Direction | Higher is better | | Results loaded | 14 of 14 | | Coverage | All results currently returned for this benchmark | Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol. ## Metrics - [Mean Accuracy](): Higher is better; unit: PERCENT. - [Best 10% Accuracy](): Higher is better; unit: PERCENT. - [Worst 10% Accuracy](): Higher is better; unit: PERCENT. - [Standard Deviation](): Lower is better; unit: SCALAR. ## Recorded results | Method | Mean Accuracy | Result date | Report paper | | --- | --- | --- | --- | | CRAFT | 70.6 | Not recorded | 6a0e91c1f984739ed6738d74 | | FedAvgM | 65.5 | Not recorded | 6a0e91c1f984739ed6738d74 | | FedAdaGrad | 64.5 | Not recorded | 6a0e91c1f984739ed6738d74 | | FedYogi | 63.9 | Not recorded | 6a0e91c1f984739ed6738d74 | | FedAdam | 63.8 | Not recorded | 6a0e91c1f984739ed6738d74 | | ConFIG | 60 | Not recorded | 6a0e91c1f984739ed6738d74 | | FedNova | 58.9 | Not recorded | 6a0e91c1f984739ed6738d74 | | FedAvg | 57 | Not recorded | 6a0e91c1f984739ed6738d74 | | FedProx | 56.5 | Not recorded | 6a0e91c1f984739ed6738d74 | | qFedAvg | 55.5 | Not recorded | 6a0e91c1f984739ed6738d74 | | AFL | 54.8 | Not recorded | 6a0e91c1f984739ed6738d74 | | FedFV | 48.9 | Not recorded | 6a0e91c1f984739ed6738d74 | | FedLF | 42.6 | Not recorded | 6a0e91c1f984739ed6738d74 | | FedMGDA+ | 38.6 | Not recorded | 6a0e91c1f984739ed6738d74 | ### CRAFT Mean Accuracy: 70.6. Result ID: 6a0f0370fbc373592c07c11c. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Architecture":"MLP"}. This record does not assert independent reproduction. ### FedAvgM Mean Accuracy: 65.5. Result ID: 6a0f0370fbc373592c07c112. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Architecture":"MLP"}. This record does not assert independent reproduction. ### FedAdaGrad Mean Accuracy: 64.5. Result ID: 6a0f0370fbc373592c07c113. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Architecture":"MLP"}. This record does not assert independent reproduction. ### FedYogi Mean Accuracy: 63.9. Result ID: 6a0f0370fbc373592c07c115. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Architecture":"MLP"}. This record does not assert independent reproduction. ### FedAdam Mean Accuracy: 63.8. Result ID: 6a0f0370fbc373592c07c114. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Architecture":"MLP"}. This record does not assert independent reproduction. ### ConFIG Mean Accuracy: 60. Result ID: 6a0f0370fbc373592c07c11b. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Architecture":"MLP"}. This record does not assert independent reproduction. ### FedNova Mean Accuracy: 58.9. Result ID: 6a0f0370fbc373592c07c111. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Architecture":"MLP"}. This record does not assert independent reproduction. ### FedAvg Mean Accuracy: 57. Result ID: 6a0f0370fbc373592c07c10f. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Architecture":"MLP"}. This record does not assert independent reproduction. ### FedProx Mean Accuracy: 56.5. Result ID: 6a0f0370fbc373592c07c110. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Architecture":"MLP"}. This record does not assert independent reproduction. ### qFedAvg Mean Accuracy: 55.5. Result ID: 6a0f0370fbc373592c07c119. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Architecture":"MLP"}. This record does not assert independent reproduction. ### AFL Mean Accuracy: 54.8. Result ID: 6a0f0370fbc373592c07c116. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Architecture":"MLP"}. This record does not assert independent reproduction. ### FedFV Mean Accuracy: 48.9. Result ID: 6a0f0370fbc373592c07c117. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Architecture":"MLP"}. This record does not assert independent reproduction. ### FedLF Mean Accuracy: 42.6. Result ID: 6a0f0370fbc373592c07c11a. [Reporting paper]() Recorded evaluation configuration: {"Architecture":"MLP"}. This record does not assert independent reproduction. ### FedMGDA+ Mean Accuracy: 38.6. Result ID: 6a0f0370fbc373592c07c118. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Architecture":"MLP"}. This record does not assert independent reproduction. Page 1 of 1. This is an index page, not the full collection.