ToTTo
Benchmarks
Task NameDataset NameSOTA ResultTrendResults
ToTTo
52.28BLEU
18
ToTTO Logic2Text-style (test)
52.7BLEU
15
ToTTo Over (test)
0.364BLEURT
15
ToTTo Non (test)
0.116BLEURT Score
15
ToTTo All (dev)
0.24BLEURT
15
ToTTo (test)
0.24BLEURT Score
15
ToTTo full (test)
50.8BLEU
12
ToTTO Logic2Text-style (dev)
46.2BLEU
10
ToTTO official (TestN)
48.7BLEU
10
ToTTo
74.2Precision
6
ToTTo
92.7Precision
6
ToTTo
77Precision
6
ToTTo (gold set)
56.89Precision
6
ToTTO Logic2Text-style (test)
0.247BLEU
5
ToTTO Logic2Text-style (TestO)
28.9BLEU
5
ToTTO Logic2Text-style (TestN)
20.5BLEU
5
ToTTO official (test)
56.7BLEU
5
ToTTO official (dev)
49BLEU
5
Totto Non-overlap (dev)
41.5BLEU Score
5
Totto All (dev)
49.2BLEU Score
5
ToTTo (human evaluation)
89TControl
4
ToTTo (dev)
48.95BLEU
4
Totto All (test)
49.5BLEU
3
ToTTo Non-Overlap (test)
41.4BLEU
3
ToTTo (test)
14.06Accuracy
1