List your company
ResearchFormerly WizWand

Natural language to strategic temporal logic translation on NL-to-ATL translation (test)

85.6Semantic Accuracygpt-5.4
12.2831.31550.3569.385Jun 29, 2026
Updated 4mo ago

Evaluation results

MethodLinks
gpt-5.4Size=API, Training=Baseline, Prompting=Few-shot2026.0685.6
qwen-coder-7bSize=7B, Training=Fine-tuned, Prompting=Few-shot2026.0684.4
gpt-4.1Size=API, Training=Baseline, Prompting=Few-shot2026.0684.3
qwen-coder-7bSize=7B, Training=Fine-tuned, Prompting=Zero-shot2026.0683.9
phi3Size=3.8B, Training=Fine-tuned, Prompting=Few-shot2026.0681.8
qwen-3bSize=3B, Training=Fine-tuned, Prompting=Few-shot2026.0680.1
phi3Size=3.8B, Training=Fine-tuned, Prompting=Zero-shot2026.0680
qwen-3bSize=3B, Training=Fine-tuned, Prompting=Zero-shot2026.0677.8
mistral-7bSize=7B, Training=Fine-tuned, Prompting=Few-shot2026.0676.6
mistral-7bSize=7B, Training=Fine-tuned, Prompting=Zero-shot2026.0676.2
gpt-5.4Size=API, Training=Baseline, Prompting=Zero-shot2026.0669.4
gpt-4.1Size=API, Training=Baseline, Prompting=Zero-shot2026.0661.1
qwen-coder-7bSize=7B, Training=Baseline, Prompting=Few-shot2026.0646.3
phi3Size=3.8B, Training=Baseline, Prompting=Few-shot2026.0641.7
qwen-3bSize=3B, Training=Baseline, Prompting=Few-shot2026.0636.7
mistral-7bSize=7B, Training=Baseline, Prompting=Few-shot2026.0634.9
qwen-coder-7bSize=7B, Training=Baseline, Prompting=Zero-shot2026.0629.8
phi3Size=3.8B, Training=Baseline, Prompting=Zero-shot2026.0627.5
mistral-7bSize=7B, Training=Baseline, Prompting=Zero-shot2026.0623.9
qwen-3bSize=3B, Training=Baseline, Prompting=Zero-shot2026.0615.1