Human Evaluation set
Benchmarks
TaskDatasetSOTA resultTrendResults
Human Evaluation Set 200 instruction-response pairs (test)
4.38Constraint Score
8
Human Evaluation Set (100 comments) 1.0 (test)
9.92Attitude Score
8
Human Evaluation Set
69.9Item-wise Score
4
Human evaluation set Generated texts (test)
79Content Score
4
Human Evaluation Set 200 edit suggestions
4.41Similarity
3
Human Evaluation Set French short-form
64.5Audio Quality MOS
3
Human evaluation set 15 videos (test)
4.4Image Prompt Alignment
3
Human Evaluation Set
4.2Quality Score
3
Human Evaluation set en-pt 1.0 (test)
2.78Gender Agreement
3
Human Evaluation set en-pl 1.0 (test)
2.64Gender Agreement
3
Human Evaluation set en-ja 1.0 (test)
2.97Gender Agreement
3
Human Evaluation set en-hi 1.0 (test)
2.89Gender Agreement
3
Human Evaluation set en-fr 1.0 (test)
2.96Gender Agreement
3
Human Evaluation set en-ar 1.0 (test)
2.79Gender Agreement
3
Human Evaluation Set
67.74Faithfulness
2
Human Evaluation Set Tourist Guidance (test)
91.11Win Rate
2
Human Evaluation Set En-Zh
69Total Instances
2
Human Evaluation Set German short-form
73.5Audio Quality
2
Human Evaluation Set Portuguese short-form
62Audio Quality
2
Human Evaluation Set Spanish short-form
66.8Audio Quality (MOS)
2