End-to-end Translation on CW
99.9AccuracyGraFT
Evaluation Results
| Method | Links | |
|---|---|---|
| GraFTData Quantity=2000, AP Lifting Model=BERT, Translation Model=T52025.12 | 99.9 | |
| NL2TLData Quantity=2000, AP Lifting Model=GPT-4o-mini, Translation Model=T52025.12 | 98.2 | |
| GraFTData Quantity=500, AP Lifting Model=BERT, Translation Model=T52025.12 | 97.7 | |
| NL2TLData Quantity=2000, AP Lifting Model=GPT-4, Translation Model=T52025.12 | 96.7 | |
| NL2TLData Quantity=500, AP Lifting Model=GPT-4o-mini, Translation Model=T52025.12 | 93 | |
| NL2TLData Quantity=500, AP Lifting Model=GPT-4, Translation Model=T52025.12 | 91.5 | |
| NL2SpecData Quantity=-, AP Lifting Model=None, Translation Model=GPT-4o-mini2025.12 | 78.6 | |
| GraFTTraining Data=CW+GLTL2025.12 | 70 | |
| NL2LTLData Quantity=-, AP Lifting Model=None, Translation Model=GPT-4o-mini2025.12 | 69.9 | |
| Ungrounded Seq2SeqData Quantity=2000, AP Lifting Model=-, Translation Model=T52025.12 | 68.1 | |
| GraFTTraining Data=GLTL+Navi2025.12 | 63.2 | |
| NL2TLTraining Data=CW+GLTL2025.12 | 63 | |
| NL2TLTraining Data=GLTL+Navi2025.12 | 60.2 | |
| Ungrounded Seq2SeqData Quantity=500, AP Lifting Model=-, Translation Model=T52025.12 | 59.6 | |
| GraFTTraining Data=CW+Navi2025.12 | 58.8 | |
| NL2TLTraining Data=CW+Navi2025.12 | 21.2 |