# VEXP: A Low-Cost RISC-V ISA Extension for Accelerated Softmax Computation in Transformers

Source: [VEXP: A Low-Cost RISC-V ISA Extension for Accelerated Softmax Computation in Transformers](<https://www.sota2.com/research/paper/68fa77ff1e209131cea764dd>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2025-10-26T04:30:26.676Z

Authors: Run Wang, Gamze Islamoglu, Andrea Belano, Viviane Potocnik, Francesco Conti, Angelo Garofalo, Luca Benini

Published: 2025-04-15T14:28:48.000Z

## Abstract

While Transformers are dominated by Floating-Point (FP) Matrix-Multiplications, their aggressive acceleration through dedicated hardware or many-core programmable systems has shifted the performance bottleneck to non-linear functions like Softmax. Accelerating Softmax is challenging due to its non-pointwise, non-linear nature, with exponentiation as the most demanding step. To address this, we design a custom arithmetic block for Bfloat16 exponentiation leveraging a novel approximation algorithm based on Schraudolph's method, and we integrate it into the Floating-Point Unit (FPU) of the RISC-V cores of a compute cluster, through custom Instruction Set Architecture (ISA) extensions, with a negligible area overhead of 1\\%. By optimizing the software kernels to leverage the extension, we execute Softmax with 162.7$\\times$ less latency and 74.3$\\times$ less energy compared to the baseline cluster, achieving an 8.2$\\times$ performance improvement and 4.1$\\times$ higher energy efficiency for the FlashAttention-2 kernel in GPT-2 configuration. Moreover, the proposed approach enables a multi-cluster system to efficiently execute end-to-end inference of pre-trained Transformer models, such as GPT-2, GPT-3 and ViT, achieving up to 5.8$\\times$ and 3.6$\\times$ reduction in latency and energy consumption, respectively, without requiring re-training and with negligible accuracy loss.

## Recorded benchmark results

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

- [Hardware Sequence Processing Performance on Sequence length n=64 at 10 GBaud](<https://www.sota2.com/research/sota/hardware-sequence-processing-performance-on-sequence-length-n-64-at-10-gbaud.md>): Hardware Sequence Processing Performance; Sequence length n=64 at 10 GBaud; Latency (s): 2.2e-7.

## Original sources and implementation

[Original paper](<https://arxiv.org/abs/2504.11227>)

[PDF](<https://arxiv.org/pdf/2504.11227.pdf>)
