Question Answering on StrategyQA (Pearson r)
0.94Pearson rOC
Evaluation results
| Method | Links | |
|---|---|---|
| OCFormat=binarized at 0.5, Signal=paper’s primary2026.06 | 0.94 | |
| OCFormat=continuous2026.06 | 0.94 | |
| −HK=10, Format=continuous2026.06 | 0.8 | |
| naive direct-answer SCK=10, Mechanism=ablation2026.06 | 0.7 | |
| naive direct-answer SCK=3, Mechanism=ablation2026.06 | 0.67 | |
| decomposed-SCK=102026.06 | 0.47 | |
| CoT-SCK=10, Format=continuous2026.06 | 0.46 | |
| CoT-SCK=3, Format=continuous2026.06 | 0.45 | |
| RLHFlow-PRMMechanism=min over steps2026.06 | 0.28 | |
| P(True)Format=continuous, Models=9 models2026.06 | 0.04 | |
| Skywork-PRMMechanism=min over steps2026.06 | 0.02 |