Contextually grounded translation on VERIFY corpus
27.8SemEqNeuroNL2LTL
Evaluation results
| Method | Links | ||
|---|---|---|---|
| NeuroNL2LTLCategory=Neural NL-to-LTL2026.05 | 27.8 | 93.7 | |
| Claude 3.5 SonnetCategory=LLMs, Prompting=5-shot2026.05 | 15.4 | 79.8 | |
| GPT-4oCategory=LLMs, Prompting=5-shot2026.05 | 14.7 | 78.2 | |
| Gemini 1.5 ProCategory=LLMs, Prompting=5-shot2026.05 | 12.9 | 74.6 | |
| Seq2Seq-AttnCategory=Neural NL-to-LTL, Data source=our data2026.05 | 11.2 | 82.4 | |
| Claude 3.5 SonnetCategory=LLMs, Prompting=zero-shot2026.05 | 9.2 | 71.3 | |
| GPT-4oCategory=LLMs, Prompting=zero-shot2026.05 | 8.3 | 67.4 | |
| Gemini 1.5 ProCategory=LLMs, Prompting=zero-shot2026.05 | 7.1 | 63.8 | |
| NL2TLCategory=Neural NL-to-LTL2026.05 | 6.8 | 58.7 | |
| Lang2LTLCategory=Neural NL-to-LTL2026.05 | 2.1 | 41.2 | |
| CopyNet-LTLCategory=Neural NL-to-LTL2026.05 | 1.4 | 38.9 |