# SPARE: Single-Pass Annotation with Reference-Guided Evaluation for Automatic Process Supervision and Reward Modelling

Source: [SPARE: Single-Pass Annotation with Reference-Guided Evaluation for Automatic Process Supervision and Reward Modelling](<https://www.sota2.com/research/paper/68fa7ad71e209131cea8574d>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2026-03-04T05:00:19.273Z

Authors: Md Imbesat Hassan Rizvi, Xiaodan Zhu, Iryna Gurevych

Published: 2025-06-18T14:37:59.000Z

## Abstract

Process or step-wise supervision has played a crucial role in advancing complex multi-step reasoning capabilities of Large Language Models (LLMs). However, efficient, high-quality automated process annotation remains a significant challenge. To address this, we introduce Single-Pass Annotation with Reference-Guided Evaluation (SPARE), a novel structured framework that enables efficient per-step annotation by jointly aligning solution steps to reference solutions and determine its accuracy with explicit reasoning in single generation. We demonstrate SPARE's effectiveness across four diverse datasets spanning mathematical reasoning (GSM8K, MATH), multi-hop question answering (MuSiQue-Ans), and spatial reasoning (SpaRP), showing consistent improvements in two applications: (1) training Process Reward Models (PRMs) for ranking and aggregating multiple generations, and (2) fine-tuning models via offline reinforcement learning for greedy decoding. On ProcessBench, SPARE demonstrates data-efficient out-of-distribution generalization, using only $\\sim$16% of training samples compared to human-labeled and other synthetically trained baselines. Additionally, it achieves competitive performance with MCTS-based methods while offering 2.3$\\times$ speedup in terms of total token count. Manual analysis reveals complementary precision-recall characteristics with MCTS approaches, suggesting potential for ensemble methods. These results establish SPARE as a practical and scalable solution for automatic process supervision in LLM reasoning.

## Recorded benchmark results

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

- [Mathematical Reasoning on MATH (SC/ORM/PRM Metrics)](<https://www.sota2.com/research/sota/mathematical-reasoning-on-math-sc-orm-prm-metrics.md>): Mathematical Reasoning; MATH; Standard Correctness (SC): 66.6.

- [Process-level Reward Modeling on PROCESSBENCH MATH](<https://www.sota2.com/research/sota/process-level-reward-modeling-on-processbench-math.md>): Process-level Reward Modeling; PROCESSBENCH MATH; Error Rate: 6.1.

- [Process-level Reward Modeling on PROCESSBENCH Olymp.Bench](<https://www.sota2.com/research/sota/process-level-reward-modeling-on-processbench-olymp-bench.md>): Process-level Reward Modeling; PROCESSBENCH Olymp.Bench; Error: 3.3.

- [Process-level Reward Modeling on PROCESSBENCH Omni-MATH](<https://www.sota2.com/research/sota/process-level-reward-modeling-on-processbench-omni-math.md>): Process-level Reward Modeling; PROCESSBENCH Omni-MATH; Error Rate: 2.8.

## Original sources and implementation

[Original paper](<https://arxiv.org/abs/2506.15498>)

[PDF](<https://arxiv.org/pdf/2506.15498.pdf>)