Human Evaluation
Benchmarks
Task NameDataset NameSOTA ResultTrendResults
Human Evaluation (test)
53.2Motion Preference Rate
2
Human Evaluation Subset (100 samples)
93CP
2
Human Evaluation
65.5Action Following
2
Human Evaluation Subset
65Win Rate
2
20 Topics Human Evaluation
4.01Interest Level Score
2
Human Evaluation Generated Stories
4.8Consistency Score
2
Human Evaluation ChartQA & web-crawled charts (test)
3.848Informativeness Score
2
Human Evaluation 6 text generation tasks (randomly sampled 10 comparison pairs for every task)
0.311Fluency Win Rate
2
Human Evaluation 100 dialogue contexts (test)
19Fluency (Win)
2
Human evaluation dataset 500 texts
4.221Image Clarity
2
Human Evaluation 400 dialogues
74DS Success Rate
2
Human Evaluation 1-hop questions (test)
56Well-formed: Yes
2
Human Evaluation Query Suggestion
0.83Bad Rating Score
1
Human Evaluation multi-turn (200 items) (test)
72Adversarial Win Rate
1
Human Evaluation (200 items) single-turn (test)
62Adversarial Win Rate
1
Human Evaluation zh⇒th
—Primary metric
0
Human Evaluation en⇒zh
—Primary metric
0