Synthetic Eval Dataset
Benchmarks
Task NameDataset NameSOTA ResultTrendResults
Synthetic Eval Dataset State
0.892BLEU
6
Synthetic Eval Dataset Sequence
0.889BLEU
6
Synthetic Eval Dataset Packet
0.63BLEU
6
Synthetic Eval Dataset Graph
88.78BLEU
6
Synthetic Eval Dataset Flowchart
0.991BLEU
6
Synthetic Eval Dataset Class
97.46BLEU
6
Synthetic Eval Dataset C4
0.992BLEU
6
Synthetic Eval Dataset Block
89.52BLEU
6
Synthetic Eval Dataset State
87.93BLEU
6
Synthetic Eval Dataset Sequence
82.36BLEU
6
Synthetic Eval Dataset Packet
0.519BLEU
6
Synthetic Eval Dataset Graph
0.636BLEU
6
Synthetic Eval Dataset Flowchart
0.908BLEU
6
Synthetic Eval Dataset (Class)
0.947BLEU
6
Synthetic Eval Dataset Block
0.868BLEU
6
Synthetic Eval Dataset 1.0 (State)
93.69BLEU
6
Synthetic Eval Dataset Sequence 1.0
1BLEU
6
Synthetic Eval Dataset Packet 1.0
1BLEU
6
Synthetic Eval Dataset Graph 1.0
0.909BLEU
6
Synthetic Eval Dataset Flowchart 1.0
0.966BLEU
6
Synthetic Eval Dataset Class 1.0
0.966BLEU
6
Synthetic Eval Dataset C4 1.0
0.961BLEU
6
Synthetic Eval Dataset Block 1.0
0.899BLEU
6