# $f$-Divergence Based Classification: Beyond the Use of Cross-Entropy Source: [$f$-Divergence Based Classification: Beyond the Use of Cross-Entropy]() Collection index: [llms.txt]() Data updated: 2025-10-26T02:44:18.752Z Authors: Nicola Novello, Andrea M. Tonello Published: 2024-01-02T16:14:02.000Z ## Abstract In deep learning, classification tasks are formalized as optimization problems often solved via the minimization of the cross-entropy. However, recent advancements in the design of objective functions allow the usage of the $f$-divergence to generalize the formulation of the optimization problem for classification. We adopt a Bayesian perspective and formulate the classification task as a maximum a posteriori probability problem. We propose a class of objective functions based on the variational representation of the $f$-divergence. Furthermore, driven by the challenge of improving the state-of-the-art approach, we propose a bottom-up method that leads us to the formulation of an objective function corresponding to a novel $f$-divergence referred to as shifted log (SL). We theoretically analyze the objective functions proposed and numerically test them in three application scenarios: toy examples, image datasets, and signal detection/decoding problems. The analyzed scenarios demonstrate the effectiveness of the proposed approach and that the SL divergence achieves the highest classification accuracy in almost all the considered cases. ## Recorded benchmark results A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. - [Image Classification on CIFAR10 (test) (Run Statistics)](): Image Classification; CIFAR10 (test); Mean Accuracy: 78.06. ## Original sources and implementation [Original paper]() [PDF]()