Multimodal Reasoning on HLE
48.8AccuracyCoT2-Meta
Evaluation Results
| Method | Links | |
|---|---|---|
| CoT2-MetaStrategy=Ours (CoT2-Meta), Inference Budget=C=16, Textification=OCR+Captioning2026.03 | 48.8 | |
| ReST-MCTS*Strategy=ReST-MCTS*, Inference Budget=C=16, Textification=OCR+Captioning2026.03 | 44.5 | |
| Vanilla ToTStrategy=Vanilla ToT, Inference Budget=C=16, Textification=OCR+Captioning2026.03 | 42.1 | |
| Best-of-16Strategy=Best-of-16, Inference Budget=C=16, Textification=OCR+Captioning2026.03 | 38.4 | |
| Pass@K (Oracle)BoN=64, Policy Model=gpt-5-mini-2025-08-072025.08 | 34.5 | |
| Greedy CoTStrategy=Greedy CoT, Inference Budget=C=16, Textification=OCR+Captioning2026.03 | 32.5 | |
| Pass@K (Oracle)BoN=32, Policy Model=gpt-5-mini-2025-08-072025.08 | 31.58 | |
| Pass@K (Oracle)BoN=16, Policy Model=gpt-5-mini-2025-08-072025.08 | 26.61 | |
| Pass@K (Oracle)BoN=8, Policy Model=gpt-5-mini-2025-08-072025.08 | 23.1 | |
| Pass@K (Oracle)BoN=4, Policy Model=gpt-5-mini-2025-08-072025.08 | 16.67 | |
| Pass@K (Oracle)BoN=2, Policy Model=gpt-5-mini-2025-08-072025.08 | 14.62 | |
| VRPRMBoN=64, Policy Model=gpt-5-mini-2025-08-072025.08 | 14.04 | |
| Major@K (Baseline)BoN=64, Policy Model=gpt-5-mini-2025-08-072025.08 | 13.74 | |
| Major@K (Baseline)BoN=32, Policy Model=gpt-5-mini-2025-08-072025.08 | 13.45 | |
| VRPRMBoN=32, Policy Model=gpt-5-mini-2025-08-072025.08 | 13.45 | |
| Major@K (Baseline)BoN=16, Policy Model=gpt-5-mini-2025-08-072025.08 | 12.87 | |
| Major@K (Baseline)BoN=8, Policy Model=gpt-5-mini-2025-08-072025.08 | 12.28 | |
| VRPRMBoN=4, Policy Model=gpt-5-mini-2025-08-072025.08 | 11.99 | |
| Major@K (Baseline)BoN=4, Policy Model=gpt-5-mini-2025-08-072025.08 | 11.4 | |
| VRPRMBoN=2, Policy Model=gpt-5-mini-2025-08-072025.08 | 11.4 | |
| VRPRMBoN=16, Policy Model=gpt-5-mini-2025-08-072025.08 | 11.4 | |
| Pass@K (Oracle)BoN=1, Policy Model=gpt-5-mini-2025-08-072025.08 | 11.11 | |
| Major@K (Baseline)BoN=1, Policy Model=gpt-5-mini-2025-08-072025.08 | 11.11 | |
| Major@K (Baseline)BoN=2, Policy Model=gpt-5-mini-2025-08-072025.08 | 11.11 | |
| VisualPRM-8BBoN=1, Policy Model=gpt-5-mini-2025-08-072025.08 | 11.11 | |
| VRPRMBoN=1, Policy Model=gpt-5-mini-2025-08-072025.08 | 11.11 | |
| VRPRMBoN=8, Policy Model=gpt-5-mini-2025-08-072025.08 | 11.11 | |
| VisualPRM-8BBoN=16, Policy Model=gpt-5-mini-2025-08-072025.08 | 10.82 | |
| VisualPRM-8BBoN=32, Policy Model=gpt-5-mini-2025-08-072025.08 | 10.82 | |
| VisualPRM-8BBoN=64, Policy Model=gpt-5-mini-2025-08-072025.08 | 10.82 | |
| VisualPRM-8BBoN=2, Policy Model=gpt-5-mini-2025-08-072025.08 | 10.53 | |
| VisualPRM-8BBoN=8, Policy Model=gpt-5-mini-2025-08-072025.08 | 10.53 | |
| VisualPRM-8BBoN=4, Policy Model=gpt-5-mini-2025-08-072025.08 | 10.23 |