# ULU: A Unified Activation Function

Source: [ULU: A Unified Activation Function](<https://www.sota2.com/research/paper/68fa7a6d1e209131cea83208>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2025-10-26T05:20:02.433Z

Authors: Simin Huo

Published: 2025-08-07T06:58:22.000Z

## Abstract

We propose \\textbf{ULU}, a novel non-monotonic, piecewise activation function defined as $\\{f(x;\\alpha\_1),x\<0; f(x;\\alpha\_2),x\>=0 \\}$, where $f(x;\\alpha)=0.5x(tanh(\\alpha x)+1),\\alpha \>0$. ULU treats positive and negative inputs differently. Extensive experiments demonstrate ULU significantly outperforms ReLU and Mish across image classification and object detection tasks. Its variant Adaptive ULU (\\textbf{AULU}) is expressed as $\\{f(x;\\beta\_1^2),x\<0; f(x;\\beta\_2^2),x\>=0 \\}$, where $\\beta\_1$ and $\\beta\_2$ are learnable parameters, enabling it to adapt its response separately for positive and negative inputs. Additionally, we introduce the LIB (Like Inductive Bias) metric from AULU to quantitatively measure the inductive bias of the model.

## Recorded benchmark results

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

## Original sources and implementation

[Original paper](<https://arxiv.org/abs/2508.05073>)

[PDF](<https://arxiv.org/pdf/2508.05073.pdf>)