End-to-end translation on Navigation
99.1AccuracyGraFT
Evaluation results
| Method | Links | |
|---|---|---|
| GraFTData Quantity=2000, AP Lifting Model=BERT, Translation Model=T52025.12 | 99.1 | |
| NL2TLData Quantity=2000, AP Lifting Model=GPT-4, Translation Model=T52025.12 | 90 | |
| NL2TLData Quantity=2000, AP Lifting Model=GPT-4o-mini, Translation Model=T52025.12 | 86.7 | |
| GraFTData Quantity=500, AP Lifting Model=BERT, Translation Model=T52025.12 | 85 | |
| NL2TLData Quantity=500, AP Lifting Model=GPT-4, Translation Model=T52025.12 | 83.7 | |
| NL2TLData Quantity=500, AP Lifting Model=GPT-4o-mini, Translation Model=T52025.12 | 80.4 | |
| NL2SpecData Quantity=-, AP Lifting Model=None, Translation Model=GPT-4o-mini2025.12 | 71.6 | |
| NL2LTLData Quantity=-, AP Lifting Model=None, Translation Model=GPT-4o-mini2025.12 | 65.3 | |
| Ungrounded Seq2SeqData Quantity=2000, AP Lifting Model=-, Translation Model=T52025.12 | 56.3 | |
| Ungrounded Seq2SeqData Quantity=500, AP Lifting Model=-, Translation Model=T52025.12 | 43.4 |