End-to-end Translation on GLTL
99.8AccuracyGraFT
Evaluation Results
| Method | Links | |
|---|---|---|
| GraFTData Quantity=2000, AP Lifting Model=BERT, Translation Model=T52025.12 | 99.8 | |
| NL2TLData Quantity=2000, AP Lifting Model=GPT-4o-mini, Translation Model=T52025.12 | 97.4 | |
| NL2TLData Quantity=2000, AP Lifting Model=GPT-4, Translation Model=T52025.12 | 96.2 | |
| GraFTData Quantity=500, AP Lifting Model=BERT, Translation Model=T52025.12 | 91.5 | |
| NL2TLData Quantity=500, AP Lifting Model=GPT-4o-mini, Translation Model=T52025.12 | 83.8 | |
| NL2TLData Quantity=500, AP Lifting Model=GPT-4, Translation Model=T52025.12 | 82.6 | |
| NL2LTLData Quantity=-, AP Lifting Model=None, Translation Model=GPT-4o-mini2025.12 | 74.4 | |
| NL2SpecData Quantity=-, AP Lifting Model=None, Translation Model=GPT-4o-mini2025.12 | 68.4 | |
| GraFTTraining Data=CW+GLTL2025.12 | 56.2 | |
| Ungrounded Seq2SeqData Quantity=2000, AP Lifting Model=-, Translation Model=T52025.12 | 55.9 | |
| GraFTTraining Data=GLTL+Navi2025.12 | 48.3 | |
| Ungrounded Seq2SeqData Quantity=500, AP Lifting Model=-, Translation Model=T52025.12 | 46.8 | |
| NL2TLTraining Data=CW+GLTL2025.12 | 45.3 | |
| NL2TLTraining Data=GLTL+Navi2025.12 | 44 | |
| GraFTTraining Data=CW+Navi2025.12 | 41.1 | |
| NL2TLTraining Data=CW+Navi2025.12 | 23.1 |