NL-to-FOL Syntax Correctness on FOLIO (test)
99Syntax Correctness RateNL2LOGIC
Evaluation results
| Method | Links | |
|---|---|---|
| NL2LOGICbackbone=gemma-2-27b2026.01 | 99 | |
| NL2LOGICbackbone=llama-3.2-1b2026.01 | 99 | |
| NL2LOGICbackbone=ministral-8b2026.01 | 99 | |
| NL2LOGICbackbone=mistral-22b2026.01 | 99 | |
| NL2LOGICbackbone=gemma-2-9b2026.01 | 98 | |
| NL2LOGICbackbone=llama-3.1-8b2026.01 | 96 | |
| NL2LOGICbackbone=qwen-2.5-7b2026.01 | 94 | |
| NL2LOGICbackbone=llama-3.2-3b2026.01 | 93 | |
| NL2LOGICbackbone=gemma-2-2b2026.01 | 92 | |
| NL2LOGICbackbone=qwen-2.5-14b2026.01 | 92 | |
| NL2LOGICbackbone=qwen-2.5-1.5b2026.01 | 87 | |
| GCDbackbone=gemma-2-27b2026.01 | 81 | |
| NL2LOGICbackbone=qwen-2.5-3b2026.01 | 81 | |
| GCDbackbone=gemma-2-9b2026.01 | 77 | |
| NL2LOGICbackbone=qwen-2.5-0.5b2026.01 | 66 | |
| GCDbackbone=gemma-2-2b2026.01 | 56 | |
| GCDbackbone=qwen-2.5-14b2026.01 | 53 | |
| GCDbackbone=qwen-2.5-1.5b2026.01 | 36 | |
| GCDbackbone=llama-3.2-3b2026.01 | 29 | |
| GCDbackbone=mistral-22b2026.01 | 29 | |
| GCDbackbone=llama-3.2-1b2026.01 | 27 | |
| GCDbackbone=qwen-2.5-0.5b2026.01 | 14 | |
| GCDbackbone=llama-3.1-8b2026.01 | 13 | |
| GCDbackbone=qwen-2.5-3b2026.01 | 12 | |
| GCDbackbone=ministral-8b2026.01 | 9 | |
| GCDbackbone=qwen-2.5-7b2026.01 | 1 |