# Climate-Eval: A Comprehensive Benchmark for NLP Tasks Related to Climate Change

Source: [Climate-Eval: A Comprehensive Benchmark for NLP Tasks Related to Climate Change](<https://www.sota2.com/research/paper/68fa78c91e209131cea7a883>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2025-10-26T04:47:07.371Z

Authors: Murathan Kurfal{\\i}, Shorouq Zahra, Joakim Nivre, Gabriele Messori

Published: 2025-05-24T11:45:46.000Z

## Abstract

Climate-Eval is a comprehensive benchmark designed to evaluate natural language processing models across a broad range of tasks related to climate change. Climate-Eval aggregates existing datasets along with a newly developed news classification dataset, created specifically for this release. This results in a benchmark of 25 tasks based on 13 datasets, covering key aspects of climate discourse, including text classification, question answering, and information extraction. Our benchmark provides a standardized evaluation suite for systematically assessing the performance of large language models (LLMs) on these tasks. Additionally, we conduct an extensive evaluation of open-source LLMs (ranging from 2B to 70B parameters) in both zero-shot and few-shot settings, analyzing their strengths and limitations in the domain of climate change.

## Recorded benchmark results

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

## Original sources and implementation

[Original paper](<https://arxiv.org/abs/2505.18653>)

[PDF](<https://arxiv.org/pdf/2505.18653.pdf>)
