List your company
ResearchFormerly WizWand

Multimodal Reasoning on HLE

48.8AccuracyCoT2-Meta
8.687219.101129.51539.9289Aug 5, 2025Sep 13, 2025Oct 23, 2025Dec 1, 2025Jan 10, 2026Feb 18, 2026Mar 30, 2026
Updated 5mo ago

Evaluation results

MethodLinks
CoT2-MetaStrategy=Ours (CoT2-Meta), Inference Budget=C=16, Textification=OCR+Captioning2026.0348.8
ReST-MCTS*Strategy=ReST-MCTS*, Inference Budget=C=16, Textification=OCR+Captioning2026.0344.5
Vanilla ToTStrategy=Vanilla ToT, Inference Budget=C=16, Textification=OCR+Captioning2026.0342.1
Best-of-16Strategy=Best-of-16, Inference Budget=C=16, Textification=OCR+Captioning2026.0338.4
Pass@K (Oracle)BoN=64, Policy Model=gpt-5-mini-2025-08-072025.0834.5
Greedy CoTStrategy=Greedy CoT, Inference Budget=C=16, Textification=OCR+Captioning2026.0332.5
Pass@K (Oracle)BoN=32, Policy Model=gpt-5-mini-2025-08-072025.0831.58
Pass@K (Oracle)BoN=16, Policy Model=gpt-5-mini-2025-08-072025.0826.61
Pass@K (Oracle)BoN=8, Policy Model=gpt-5-mini-2025-08-072025.0823.1
Pass@K (Oracle)BoN=4, Policy Model=gpt-5-mini-2025-08-072025.0816.67
Pass@K (Oracle)BoN=2, Policy Model=gpt-5-mini-2025-08-072025.0814.62
VRPRMBoN=64, Policy Model=gpt-5-mini-2025-08-072025.0814.04
Major@K (Baseline)BoN=64, Policy Model=gpt-5-mini-2025-08-072025.0813.74
Major@K (Baseline)BoN=32, Policy Model=gpt-5-mini-2025-08-072025.0813.45
VRPRMBoN=32, Policy Model=gpt-5-mini-2025-08-072025.0813.45
Major@K (Baseline)BoN=16, Policy Model=gpt-5-mini-2025-08-072025.0812.87
Major@K (Baseline)BoN=8, Policy Model=gpt-5-mini-2025-08-072025.0812.28
VRPRMBoN=4, Policy Model=gpt-5-mini-2025-08-072025.0811.99
Major@K (Baseline)BoN=4, Policy Model=gpt-5-mini-2025-08-072025.0811.4
VRPRMBoN=2, Policy Model=gpt-5-mini-2025-08-072025.0811.4
VRPRMBoN=16, Policy Model=gpt-5-mini-2025-08-072025.0811.4
Pass@K (Oracle)BoN=1, Policy Model=gpt-5-mini-2025-08-072025.0811.11
Major@K (Baseline)BoN=1, Policy Model=gpt-5-mini-2025-08-072025.0811.11
Major@K (Baseline)BoN=2, Policy Model=gpt-5-mini-2025-08-072025.0811.11
VisualPRM-8BBoN=1, Policy Model=gpt-5-mini-2025-08-072025.0811.11
VRPRMBoN=1, Policy Model=gpt-5-mini-2025-08-072025.0811.11
VRPRMBoN=8, Policy Model=gpt-5-mini-2025-08-072025.0811.11
VisualPRM-8BBoN=16, Policy Model=gpt-5-mini-2025-08-072025.0810.82
VisualPRM-8BBoN=32, Policy Model=gpt-5-mini-2025-08-072025.0810.82
VisualPRM-8BBoN=64, Policy Model=gpt-5-mini-2025-08-072025.0810.82
VisualPRM-8BBoN=2, Policy Model=gpt-5-mini-2025-08-072025.0810.53
VisualPRM-8BBoN=8, Policy Model=gpt-5-mini-2025-08-072025.0810.53
VisualPRM-8BBoN=4, Policy Model=gpt-5-mini-2025-08-072025.0810.23