High-level Planning research benchmarks

Benchmarks

Dataset NameSOTA methodMetricTrendResultsLast Updated

VLABench vlm_evaluation 1.0 (test)

Oracle

85.8AUC

Jun 11, 2026

NAVSIM

ELF-VLA

85.8Speed Accuracy

Mar 4, 2026

RoboTwin Hard

RoboPARA

2.4TEI

Mar 4, 2026

RoboTwin Medium

LLM3

2.1Task Execution Index (TEI)

Mar 4, 2026

RoboTwin Easy

RoboPARA

10.6Task Execution Index (TEI)

Mar 4, 2026

Simulated Tasks All tasks

Gemini

86.1Success Rate

Mar 4, 2026

Simulated Tasks >7 actions (Long split)

Gemini

65.18Success Rate

Mar 4, 2026

Simulated Tasks Medium 3–7 actions

Gemini

97.96Success Rate

Mar 4, 2026

Simulated Tasks ≤2 actions (Short)

Gemini

97.19Success Rate

Mar 4, 2026

ALFRED (val unseen)

Alfred

64EM

Feb 26, 2026

ReasonMap L (long questions)

Ariadne

0.075Weighted Accuracy

Apr 15, 2026

ReasonMap S (short questions)

SFT VLM

15.44Weighted Accuracy

Apr 15, 2026