# Mask-Proof: An LLM-based Automated Data Curation Pipeline on Mathematical Proofs

Source: [Mask-Proof: An LLM-based Automated Data Curation Pipeline on Mathematical Proofs](<https://www.sota2.com/research/paper/6a30d8bfd33dfb23e783cdad>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2026-06-17T05:00:05.807Z

Authors: Jierui Zhang, Siyuan Tan, Xinhang Li, Longzhuangzhi Lin, Dailin Li, Chengfeng Gu, Xinping Li, Yaxian Hao, Shengjia Liang, Yuxiang Ren, Wenhao Liu

Published: 2026-06-13T11:26:09.000Z

## Abstract

Large language models (LLMs) are increasingly capable of mathematical problem solving and can even assist with research-level proofs, yet we still lack a scalable and reproducible way to measure step-level reasoning in long proofs across diverse sources. This evaluation gap limits trustworthy AI assistance in proof-certified scientific progress. Existing evaluations often emphasize final answers or rely on costly expert grading, while end-to-end proof generation remains open-ended and hard to verify automatically. We introduce Mask-Proof, a pipeline that turns real proofs into automatically checkable masked-step tasks. It masks key formula steps, provides the necessary surrounding context, and evaluates model reconstructions with an LLM-based equivalence judge using repeated votes for stability. The resulting Mask-ProofBench contains 292 curated problems across diverse research areas. Experiments with 17 models show that reasoning-enhanced models outperform standard models by 12% to 27%. Our evaluator achieves 96.8% agreement with expert annotators, enabling faithful, reproducible, and comparable measurement of step-level mathematical reasoning. Benchmark, annotations, and code are available at https://github.com/weating/Mask-Proof.

## Recorded benchmark results

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

- [Mathematical Proof Reasoning on IMO-ProofBench Advanced](<https://www.sota2.com/research/sota/mathematical-proof-reasoning-on-imo-proofbench-advanced.md>): Mathematical Proof Reasoning; IMO-ProofBench Advanced (full set); Mask-Proof Average@4: not recorded.

- [Mathematical Proof Reasoning on IMO-ProofBench-Advanced Novel (22 problems)](<https://www.sota2.com/research/sota/mathematical-proof-reasoning-on-imo-proofbench-advanced-novel-22-problems.md>): Mathematical Proof Reasoning; IMO-ProofBench-Advanced Novel (22 problems); Mask-Proof Avg@4: not recorded.

- [Mathematical Proof Reasoning on IMO-ProofBench-Advanced USAMO 2025 (5 problems)](<https://www.sota2.com/research/sota/mathematical-proof-reasoning-on-imo-proofbench-advanced-usamo-2025-5-problems.md>): Mathematical Proof Reasoning; IMO-ProofBench-Advanced USAMO 2025 (5 problems); Mask-Proof Avg@4: not recorded.

## Original sources and implementation

[Original paper](<https://arxiv.org/abs/2606.15258>)

[PDF](<https://arxiv.org/pdf/2606.15258.pdf>)
