Categories Companies Research

Loading the SOTA2 catalog…

Explore

Categories Companies Research

Contribute

Join SOTA2 Contact

SOTA2

© 2026 SOTA2

Build in Bellevue, USA

WizWand is now SOTA2 Research

Papers Benchmarks Datasets Tasks

Semantic Parsing research benchmarks – Page 3 · SOTA2 Research

Semantic Parsing

Benchmarks

Dataset NameSOTA MethodSort

Dataset NameSOTA methodMetricTrendResultsLast Updated

SMCalFlow CS (i.i.d.)

T5 (fine tuned w/o prompts)

88.5Exact Match Accuracy

8

Feb 26, 2026

GeoQuery (Templ.)

T5 (fine tuned w/o prompts)

85.9Exact Match Accuracy

8

Feb 26, 2026

SMCalFlow-CS (32-C)

70.4Accuracy

8

Feb 26, 2026

SMCalFlow CS (8-C)

Cover-LS (Oracle)

77.3Exact Match Accuracy

8

Feb 26, 2026

83.1EM

8

Feb 26, 2026

PMB English 3.0.0 (test)

89.3F1 Score

8

Feb 26, 2026

PMB English 3.0.0 (dev)

88.4F1 Score

8

Feb 26, 2026

90.4Accuracy

8

Feb 26, 2026

UCCA 20K Leagues set out-of-domain (test)

68.7Primary Labeled Precision

8

Feb 26, 2026

91.3Accuracy

8

Feb 26, 2026

LoRA-r16 + replay 10%

95.84COGS EM

7

Jul 1, 2026

SMCalFlow-CS 32-C v1 & v2

Cover-LS (NoFT)

64.7Accuracy (v2)

7

Feb 26, 2026

SMCalFlow-CS 16-C v1 & v2

Cover-LS (NoFT)

60.3Accuracy v2

7

Feb 26, 2026

SMCalFlow-CS v1 & v2 (i.i.d.)

T5 Large (fine tuned w/o prompts)

83.3Accuracy (v2)

7

Feb 26, 2026

ATIS length split

Multiset Tagging and Latent Permutations

41.39Accuracy

7

Feb 26, 2026

86.96Accuracy

7

Feb 26, 2026

92.36Average Score

7

Feb 26, 2026

GEOQUERY 280-sentence Geo880 (test)

96.3Precision

7

Feb 26, 2026

92.1MCD1 Score

6

May 7, 2026

100Accuracy

6

Feb 26, 2026

100Accuracy

6

Feb 26, 2026

100Accuracy

6

Feb 26, 2026

SCAN 0-shot lexical

99Accuracy (0-shot)

6

Feb 26, 2026

SCAN 1-shot lexical

100Accuracy

6

Feb 26, 2026

100Accuracy

6

Feb 26, 2026

Page 3 of 5

Popular datasets

GEOQUERY SCAN CFQ PMB SMCalFlow

Follow for updates

Join SOTA2 Contact