Categories Companies Research

Loading the SOTA2 catalog…

Explore

Categories Companies Research

Contribute

Join SOTA2 Contact

SOTA2

© 2026 SOTA2

Build in Bellevue, USA

WizWand is now SOTA2 Research

Papers Benchmarks Datasets Tasks

User Simulation research benchmarks · SOTA2 Research

User Simulation

Benchmarks

Dataset NameSOTA MethodSort

Dataset NameSOTA methodMetricTrendResultsLast Updated

Synthetic Exposure Overall

Llama-3.2-3B-Instruct +SFT+DPO

55Accuracy (%)

19

Apr 20, 2026

MIND Real Exposure

Gemini-3.0-Pro-Preview (2025-11-18)

34.8Accuracy

19

Apr 20, 2026

τ-USI retail airline tasks (out-of-distribution)

Human (upper bound)

87.4Conv Score

16

Jun 15, 2026

93Primary Metric Score

15

Jun 15, 2026

38.12F1 Score

13

Apr 14, 2026

29.74F1 Score

13

Apr 14, 2026

31.45F1 Score

13

Apr 14, 2026

Adversarial User Simulation Dataset Turn-level (test)

95Robotics Score

11

Feb 26, 2026

User Simulation Dataset Session-level (test)

95.21Role Score

11

Feb 26, 2026

WildChat (In-Distribution)

Prompted Simulator

8.62Turn Count

8

May 13, 2026

0.713Realism Score (LLM-judge)

6

May 21, 2026

ConvApparel Efficient Matchmaker Ultra-terse Fast V2 (held-out agent)

Static, Agnostic

25.4Avg. Words per Turn

5

May 13, 2026

ConvApparel Domain Expert Academic Verbose V2 (held-out agent)

12.1Average Words per Turn

5

May 13, 2026

HUMANUAL (test)

40.58News Score

3

Mar 5, 2026

Popular datasets

ConvApparel MovieLens LastFM MIND WildChat

Follow for updates

Join SOTA2 Contact