# Towards Accurate and Reliable Energy Measurement of NLP Models

Source: [Towards Accurate and Reliable Energy Measurement of NLP Models](<https://www.sota2.com/research/paper/68fa68861e209131cea1cace>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2025-10-25T22:39:59.706Z

Authors: Qingqing Cao, Aruna Balasubramanian, Niranjan Balasubramanian

Published: 2020-10-11T13:44:52.000Z

## Abstract

Accurate and reliable measurement of energy consumption is critical for making well-informed design choices when choosing and training large scale NLP models. In this work, we show that existing software-based energy measurements are not accurate because they do not take into account hardware differences and how resource utilization affects energy consumption. We conduct energy measurement experiments with four different models for a question answering task. We quantify the error of existing software-based energy measurements by using a hardware power meter that provides highly accurate energy measurements. Our key takeaway is the need for a more accurate energy estimation model that takes into account hardware variabilities and the non-linear relationship between resource utilization and energy consumption. We release the code and data at https://github.com/csarron/sustainlp2020-energy.

## Recorded benchmark results

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

## Original sources and implementation

[Original paper](<https://arxiv.org/abs/2010.05248>)

[PDF](<https://arxiv.org/pdf/2010.05248.pdf>)
