# Step-level reasoning verification on Calendar Source: [Step-level reasoning verification on Calendar]() Collection index: [llms.txt]() Data updated: 2026-04-23T10:01:13.396Z Evaluation of model performance on the Step-level reasoning verification task using the Calendar dataset, measured by PR-AUC. A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. | Scope | Recorded value | | --- | --- | | Task | Step-level reasoning verification | | Dataset (including any stated split) | Calendar | | Metric | PR-AUC | | Unit | PERCENT | | Direction | Higher is better | | Results loaded | 19 of 19 | | Coverage | All results currently returned for this benchmark | Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol. ## Metrics - [PR-AUC](): Higher is better; unit: PERCENT. ## Recorded results | Method | PR-AUC | Result date | Report paper | | --- | --- | --- | --- | | ReProbe, Hidden States, GPT-OSS-anno | 69.8 | Not recorded | 6912c2ee2828b7cc6937d9ec | | Qwen2.5-Math-PRM-7B | 67.6 | Not recorded | 6912c2ee2828b7cc6937d9ec | | Universal-PRM-Qwen2.5-Math-7B | 63.7 | Not recorded | 6912c2ee2828b7cc6937d9ec | | Qwen2.5-Math-7B-PRM800K | 59.4 | Not recorded | 6912c2ee2828b7cc6937d9ec | | ReProbe, Attn+Logit, Qwen3-8B-anno | 59.2 | Not recorded | 6912c2ee2828b7cc6937d9ec | | Qwen2.5-Math-7B-PRM800k | 56.8 | Not recorded | 6912c2ee2828b7cc6937d9ec | | Qwen2.5-Math-7B | 55.5 | Not recorded | 6912c2ee2828b7cc6937d9ec | | Skywork-PRM-1.5B | 50.8 | Not recorded | 6912c2ee2828b7cc6937d9ec | | Universal-PRM-Qwen2.5-Math-7B | 49.7 | Not recorded | 6912c2ee2828b7cc6937d9ec | | Math-Shepherd-PRM-7B | 48.9 | Not recorded | 6912c2ee2828b7cc6937d9ec | | H4-Qwen2.5-PRM-1.5B-0.2 | 43.4 | Not recorded | 6912c2ee2828b7cc6937d9ec | | H4-Qwen2.5-PRM-1.5B-0.2 | 38.6 | Not recorded | 6912c2ee2828b7cc6937d9ec | | MaxProb | 38 | Not recorded | 6912c2ee2828b7cc6937d9ec | | MaxEntropy | 36.2 | Not recorded | 6912c2ee2828b7cc6937d9ec | | RLHFlow-PRM-Deepseek-8B | 34.3 | Not recorded | 6912c2ee2828b7cc6937d9ec | | Skywork-PRM-1.5B | 32.7 | Not recorded | 6912c2ee2828b7cc6937d9ec | | Random | 32.4 | Not recorded | 6912c2ee2828b7cc6937d9ec | | Perplexity | 32.3 | Not recorded | 6912c2ee2828b7cc6937d9ec | | RLHFlow-PRM-Mistral-8B | 30.9 | Not recorded | 6912c2ee2828b7cc6937d9ec | ### ReProbe, Hidden States, GPT-OSS-anno PR-AUC: 69.8. Result ID: 69e9edd747f34ce924d19e28. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"# Sample":"10.8K","Verifying Model":"Qwen3-32B","Thinking Mode":"native"}. This record does not assert independent reproduction. ### Qwen2.5-Math-PRM-7B PR-AUC: 67.6. Result ID: 69e9edd747f34ce924d19e27. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"# Sample":"860K","Verifying Model":"Qwen3-32B","Thinking Mode":"native"}. This record does not assert independent reproduction. ### Universal-PRM-Qwen2.5-Math-7B PR-AUC: 63.7. Result ID: 69e9edd747f34ce924d19e25. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"# Sample":"690K","Verifying Model":"Qwen3-32B","Thinking Mode":"native"}. This record does not assert independent reproduction. ### Qwen2.5-Math-7B-PRM800K PR-AUC: 59.4. Result ID: 69e9edd747f34ce924d19e26. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"# Sample":"263K","Verifying Model":"Qwen3-32B","Thinking Mode":"native"}. This record does not assert independent reproduction. ### ReProbe, Attn+Logit, Qwen3-8B-anno PR-AUC: 59.2. Result ID: 69e9edd747f34ce924d19ced. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"# Sample":"32K"}. This record does not assert independent reproduction. ### Qwen2.5-Math-7B-PRM800k PR-AUC: 56.8. Result ID: 69e9edd747f34ce924d19ceb. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"# Sample":"265K"}. This record does not assert independent reproduction. ### Qwen2.5-Math-7B PR-AUC: 55.5. Result ID: 69e9edd747f34ce924d19cec. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"# Sample":"860K"}. This record does not assert independent reproduction. ### Skywork-PRM-1.5B PR-AUC: 50.8. Result ID: 69e9edd747f34ce924d19e23. [Reporting paper]() Recorded evaluation configuration: {"# Sample":"Unk","Verifying Model":"Qwen3-32B","Thinking Mode":"native"}. This record does not assert independent reproduction. ### Universal-PRM-Qwen2.5-Math-7B PR-AUC: 49.7. Result ID: 69e9edd747f34ce924d19cea. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"# Sample":"690K"}. This record does not assert independent reproduction. ### Math-Shepherd-PRM-7B PR-AUC: 48.9. Result ID: 69e9edd747f34ce924d19ce7. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"# Sample":"440K"}. This record does not assert independent reproduction. ### H4-Qwen2.5-PRM-1.5B-0.2 PR-AUC: 43.4. Result ID: 69e9edd747f34ce924d19ce6. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"# Sample":"369K"}. This record does not assert independent reproduction. ### H4-Qwen2.5-PRM-1.5B-0.2 PR-AUC: 38.6. Result ID: 69e9edd747f34ce924d19e24. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"# Sample":"369K","Verifying Model":"Qwen3-32B","Thinking Mode":"native"}. This record does not assert independent reproduction. ### MaxProb PR-AUC: 38. Result ID: 69e9edd747f34ce924d19ce2. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"# Sample":"-"}. This record does not assert independent reproduction. ### MaxEntropy PR-AUC: 36.2. Result ID: 69e9edd747f34ce924d19ce3. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"# Sample":"-"}. This record does not assert independent reproduction. ### RLHFlow-PRM-Deepseek-8B PR-AUC: 34.3. Result ID: 69e9edd747f34ce924d19ce8. [Reporting paper]() Recorded evaluation configuration: {"# Sample":"253K"}. This record does not assert independent reproduction. ### Skywork-PRM-1.5B PR-AUC: 32.7. Result ID: 69e9edd747f34ce924d19ce5. [Reporting paper]() Recorded evaluation configuration: {"# Sample":"Unk"}. This record does not assert independent reproduction. ### Random PR-AUC: 32.4. Result ID: 69e9edd747f34ce924d19ce1. [Reporting paper]() Recorded evaluation configuration: {"# Sample":"-"}. This record does not assert independent reproduction. ### Perplexity PR-AUC: 32.3. Result ID: 69e9edd747f34ce924d19ce4. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"# Sample":"-"}. This record does not assert independent reproduction. ### RLHFlow-PRM-Mistral-8B PR-AUC: 30.9. Result ID: 69e9edd747f34ce924d19ce9. [Reporting paper]() Recorded evaluation configuration: {"# Sample":"273K"}. This record does not assert independent reproduction. Page 1 of 1. This is an index page, not the full collection.