Categories Companies Research

Loading the SOTA2 catalog…

Explore

Categories Companies Research

Contribute

Join SOTA2 Contact

SOTA2

© 2026 SOTA2

Build in Bellevue, USA

WizWand is now SOTA2 Research

Papers Benchmarks Datasets Tasks

LEMMA dataset and benchmarks · SOTA2 Research

Research Datasets

LEMMA

Benchmarks

Task NameDataset NameSOTA MethodSort

Task NameDataset NameSOTA ResultTrendResults

Robotic Planning

LEMMA Single-Agent Overall

80Success Rate (SR)

14

Robotic Planning

LEMMA Single-Agent Underspecified

99Success Rate (SR)

14

Robotic Planning

LEMMA Single-Agent Absence

96Success Rate (SR)

14

Robotic Planning

LEMMA Single-Agent Multiplicity

77Success Rate (SR)

14

Robotic Planning

LEMMA Stack and Pass tasks, partially observed (test)

75Success Rate

8

Egocentric latent state prediction

LEMMA

0.058L2 Error (2s)

7

Keystep recognition

LEMMA

27.86Top-1 Accuracy

4

Temporal Grounding

LEMMA D views

18Recall@1

4

Robotic Task Planning

LEMMA Single-agent

10.36Calls per Episode

4

Fault detection

LEMMA

74.1AUROC

2

Popular tasks

Robotic Planning Temporal Grounding Keystep recognition Robotic Task Planning Fault Detection

Follow for updates

Join SOTA2 Contact