# Exploring the Potential of Flexible 8-bit Format: Design and Algorithm

Source: [Exploring the Potential of Flexible 8-bit Format: Design and Algorithm](<https://www.sota2.com/research/paper/68fa710a1e209131cea4ed18>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2025-10-26T01:56:42.060Z

Authors: Zhuoyi Zhang, Yunchen Zhang, Gonglei Shi, Yu Shen, Ruihao Gong, Xiaoxu Xia, Qi Zhang, Lewei Lu, Xianglong Liu

Published: 2023-10-20T13:57:28.000Z

## Abstract

Neural network quantization is widely used to reduce model inference complexity in real-world deployments. However, traditional integer quantization suffers from accuracy degradation when adapting to various dynamic ranges. Recent research has focused on a new 8-bit format, FP8, with hardware support for both training and inference of neural networks but lacks guidance for hardware design. In this paper, we analyze the benefits of using FP8 quantization and provide a comprehensive comparison of FP8 with INT quantization. Then we propose a flexible mixed-precision quantization framework that supports various number systems, enabling optimal selection of the most appropriate quantization format for different neural network architectures. Experimental results demonstrate that our proposed framework achieves competitive performance compared to full precision on various tasks, including image classification, object detection, segmentation, and natural language understanding. Our work furnishes critical insights into the tangible benefits and feasibility of employing FP8 quantization, paving the way for heightened neural network efficiency in tangible scenarios. Our code is available in the supplementary material.

## Recorded benchmark results

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

## Original sources and implementation

[Original paper](<https://arxiv.org/abs/2310.13513>)

[PDF](<https://arxiv.org/pdf/2310.13513.pdf>)
