List your company
ResearchFormerly WizWand

Step-level correctness assessment on Calendar (test)

0.789PR-AUCReProbe
0.403160.503330.60350.70367Nov 9, 2025
Updated 6mo ago

Evaluation results

MethodLinks
ReProbe# Sample=32K, Architecture=Hidden States, Annotation=DeepSeek-anno2025.110.789
ReProbe# Sample=32K, Architecture=Attn+Logit, Annotation=DeepSeek-anno2025.110.786
ReProbe# Sample=32K, Architecture=Hidden States, Annotation=Self-anno2025.110.785
ReProbe# Sample=32K, Architecture=Attn+Logit, Annotation=Self-anno2025.110.779
Qwen2.5-Math-PRM-7B# Sample=860K2025.110.745
Qwen2.5-Math-7B-PRM800k# Sample=265K2025.110.727
Universal-PRM-Qwen2.5-Math-7B# Sample=690K2025.110.691
Math-Shepherd-PRM-7B# Sample=440K2025.110.657
RLHFlow-PRM-Deepseek-8B# Sample=253K2025.110.504
Skywork-PRM-1.5B# Sample=Unk2025.110.502
RLHFlow-PRM-Mistral-8B# Sample=273K2025.110.499
H4-Qwen2.5-PRM-1.5B-0.2# Sample=369K2025.110.495
Semantic Entropy# Sample=-2025.110.492
P(True)# Sample=-2025.110.49
Lexical Similarity# Sample=-2025.110.49
Random# Sample=-2025.110.486
Degree Matrix# Sample=-2025.110.484
MaxProb# Sample=-2025.110.483
Self-Certainty# Sample=-2025.110.482
CCP# Sample=-2025.110.452
MaxEntropy# Sample=-2025.110.443
Perplexity# Sample=-2025.110.418