# Don't Fear Peculiar Activation Functions: EUAF and Beyond

Source: [Don't Fear Peculiar Activation Functions: EUAF and Beyond](<https://www.sota2.com/research/paper/68fa740f1e209131cea5f8d2>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2026-06-26T05:00:12.659Z

Authors: Qianchao Wang, Shijun Zhang, Dong Zeng, Zhaoheng Xie, Hengtao Guo, Feng-Lei Fan, Tieyong Zeng (1) \_\_INSTITUTION\_7\_\_ Center of Mathematical Artificial Intelligence, Department of Mathematics, The Chinese University of Hong Kong, Hong Kong, China, (2) Department of Applied Mathematics, The Hong Kong Polytechnic University, Hong Kong, China, (3) Department of Biomedical Engineering, Southern Medical University, Guangzhou, China, (4) Institute of Medical Technology, Peking University Health Science Center, Peking University, Beijing, China, (5) Independent Researcher, Seattle, WA, US)

Published: 2024-07-12T03:57:25.000Z

## Abstract

In this paper, we propose a new super-expressive activation function called the Parametric Elementary Universal Activation Function (PEUAF). We demonstrate the effectiveness of PEUAF through systematic and comprehensive experiments on various industrial and image datasets, including CIFAR10, Tiny-ImageNet, and ImageNet. Moreover, we significantly generalize the family of super-expressive activation functions, whose existence has been demonstrated in several recent works by showing that any continuous function can be approximated to any desired accuracy by a fixed-size network with a specific super-expressive activation function. Specifically, our work addresses two major bottlenecks in impeding the development of super-expressive activation functions: the limited identification of super-expressive functions, which raises doubts about their broad applicability, and their often peculiar forms, which lead to skepticism regarding their scalability and practicality in real-world applications.

## Recorded benchmark results

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

- [Image Classification on CIFAR-10 (test) (Acc and AA)](<https://www.sota2.com/research/sota/image-classification-on-cifar-10-test-acc-and-aa.md>): Image Classification; CIFAR-10 (test); Accuracy: not recorded.

## Original sources and implementation

[Original paper](<https://arxiv.org/abs/2407.09580>)

[PDF](<https://arxiv.org/pdf/2407.09580.pdf>)