# NL-to-FOL Syntax Correctness on FOLIO (test) Source: [NL-to-FOL Syntax Correctness on FOLIO (test)]() Collection index: [llms.txt]() Data updated: 2026-02-26T06:05:48.283Z Evaluation of model performance on assessing the syntactic correctness of Natural Language (NL) translations into First-Order Logic (FOL) using the test split of the FOLIO dataset, measured by Syntax Correctness Rate. A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. | Scope | Recorded value | | --- | --- | | Task | NL-to-FOL Syntax Correctness | | Dataset (including any stated split) | FOLIO (test) | | Metric | Syntax Correctness Rate | | Unit | PERCENT | | Direction | Higher is better | | Results loaded | 26 of 26 | | Coverage | All results currently returned for this benchmark | Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol. ## Metrics - [Syntax Correctness Rate](): Higher is better; unit: PERCENT. ## Recorded results | Method | Syntax Correctness Rate | Result date | Report paper | | --- | --- | --- | --- | | NL2LOGIC | 99 | Not recorded | 6993f6239ebc499c97409b3a | | NL2LOGIC | 99 | Not recorded | 6993f6239ebc499c97409b3a | | NL2LOGIC | 99 | Not recorded | 6993f6239ebc499c97409b3a | | NL2LOGIC | 99 | Not recorded | 6993f6239ebc499c97409b3a | | NL2LOGIC | 98 | Not recorded | 6993f6239ebc499c97409b3a | | NL2LOGIC | 96 | Not recorded | 6993f6239ebc499c97409b3a | | NL2LOGIC | 94 | Not recorded | 6993f6239ebc499c97409b3a | | NL2LOGIC | 93 | Not recorded | 6993f6239ebc499c97409b3a | | NL2LOGIC | 92 | Not recorded | 6993f6239ebc499c97409b3a | | NL2LOGIC | 92 | Not recorded | 6993f6239ebc499c97409b3a | | NL2LOGIC | 87 | Not recorded | 6993f6239ebc499c97409b3a | | GCD | 81 | Not recorded | 6993f6239ebc499c97409b3a | | NL2LOGIC | 81 | Not recorded | 6993f6239ebc499c97409b3a | | GCD | 77 | Not recorded | 6993f6239ebc499c97409b3a | | NL2LOGIC | 66 | Not recorded | 6993f6239ebc499c97409b3a | | GCD | 56 | Not recorded | 6993f6239ebc499c97409b3a | | GCD | 53 | Not recorded | 6993f6239ebc499c97409b3a | | GCD | 36 | Not recorded | 6993f6239ebc499c97409b3a | | GCD | 29 | Not recorded | 6993f6239ebc499c97409b3a | | GCD | 29 | Not recorded | 6993f6239ebc499c97409b3a | | GCD | 27 | Not recorded | 6993f6239ebc499c97409b3a | | GCD | 14 | Not recorded | 6993f6239ebc499c97409b3a | | GCD | 13 | Not recorded | 6993f6239ebc499c97409b3a | | GCD | 12 | Not recorded | 6993f6239ebc499c97409b3a | | GCD | 9 | Not recorded | 6993f6239ebc499c97409b3a | | GCD | 1 | Not recorded | 6993f6239ebc499c97409b3a | ### NL2LOGIC Syntax Correctness Rate: 99. Result ID: 699a70b15698665ec4f976fc. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"backbone":"gemma-2-27b"}. This record does not assert independent reproduction. ### NL2LOGIC Syntax Correctness Rate: 99. Result ID: 699a70b15698665ec4f976fe. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"backbone":"llama-3.2-1b"}. This record does not assert independent reproduction. ### NL2LOGIC Syntax Correctness Rate: 99. Result ID: 699a70b15698665ec4f97704. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"backbone":"ministral-8b"}. This record does not assert independent reproduction. ### NL2LOGIC Syntax Correctness Rate: 99. Result ID: 699a70b15698665ec4f97706. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"backbone":"mistral-22b"}. This record does not assert independent reproduction. ### NL2LOGIC Syntax Correctness Rate: 98. Result ID: 699a70b15698665ec4f976fa. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"backbone":"gemma-2-9b"}. This record does not assert independent reproduction. ### NL2LOGIC Syntax Correctness Rate: 96. Result ID: 699a70b15698665ec4f97702. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"backbone":"llama-3.1-8b"}. This record does not assert independent reproduction. ### NL2LOGIC Syntax Correctness Rate: 94. Result ID: 699a70b15698665ec4f9770e. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"backbone":"qwen-2.5-7b"}. This record does not assert independent reproduction. ### NL2LOGIC Syntax Correctness Rate: 93. Result ID: 699a70b15698665ec4f97700. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"backbone":"llama-3.2-3b"}. This record does not assert independent reproduction. ### NL2LOGIC Syntax Correctness Rate: 92. Result ID: 699a70b15698665ec4f976f8. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"backbone":"gemma-2-2b"}. This record does not assert independent reproduction. ### NL2LOGIC Syntax Correctness Rate: 92. Result ID: 699a70b15698665ec4f97710. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"backbone":"qwen-2.5-14b"}. This record does not assert independent reproduction. ### NL2LOGIC Syntax Correctness Rate: 87. Result ID: 699a70b15698665ec4f9770a. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"backbone":"qwen-2.5-1.5b"}. This record does not assert independent reproduction. ### GCD Syntax Correctness Rate: 81. Result ID: 699a70b15698665ec4f976fb. [Reporting paper]() Recorded evaluation configuration: {"backbone":"gemma-2-27b"}. This record does not assert independent reproduction. ### NL2LOGIC Syntax Correctness Rate: 81. Result ID: 699a70b15698665ec4f9770c. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"backbone":"qwen-2.5-3b"}. This record does not assert independent reproduction. ### GCD Syntax Correctness Rate: 77. Result ID: 699a70b15698665ec4f976f9. [Reporting paper]() Recorded evaluation configuration: {"backbone":"gemma-2-9b"}. This record does not assert independent reproduction. ### NL2LOGIC Syntax Correctness Rate: 66. Result ID: 699a70b15698665ec4f97708. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"backbone":"qwen-2.5-0.5b"}. This record does not assert independent reproduction. ### GCD Syntax Correctness Rate: 56. Result ID: 699a70b15698665ec4f976f7. [Reporting paper]() Recorded evaluation configuration: {"backbone":"gemma-2-2b"}. This record does not assert independent reproduction. ### GCD Syntax Correctness Rate: 53. Result ID: 699a70b15698665ec4f9770f. [Reporting paper]() Recorded evaluation configuration: {"backbone":"qwen-2.5-14b"}. This record does not assert independent reproduction. ### GCD Syntax Correctness Rate: 36. Result ID: 699a70b15698665ec4f97709. [Reporting paper]() Recorded evaluation configuration: {"backbone":"qwen-2.5-1.5b"}. This record does not assert independent reproduction. ### GCD Syntax Correctness Rate: 29. Result ID: 699a70b15698665ec4f976ff. [Reporting paper]() Recorded evaluation configuration: {"backbone":"llama-3.2-3b"}. This record does not assert independent reproduction. ### GCD Syntax Correctness Rate: 29. Result ID: 699a70b15698665ec4f97705. [Reporting paper]() Recorded evaluation configuration: {"backbone":"mistral-22b"}. This record does not assert independent reproduction. ### GCD Syntax Correctness Rate: 27. Result ID: 699a70b15698665ec4f976fd. [Reporting paper]() Recorded evaluation configuration: {"backbone":"llama-3.2-1b"}. This record does not assert independent reproduction. ### GCD Syntax Correctness Rate: 14. Result ID: 699a70b15698665ec4f97707. [Reporting paper]() Recorded evaluation configuration: {"backbone":"qwen-2.5-0.5b"}. This record does not assert independent reproduction. ### GCD Syntax Correctness Rate: 13. Result ID: 699a70b15698665ec4f97701. [Reporting paper]() Recorded evaluation configuration: {"backbone":"llama-3.1-8b"}. This record does not assert independent reproduction. ### GCD Syntax Correctness Rate: 12. Result ID: 699a70b15698665ec4f9770b. [Reporting paper]() Recorded evaluation configuration: {"backbone":"qwen-2.5-3b"}. This record does not assert independent reproduction. ### GCD Syntax Correctness Rate: 9. Result ID: 699a70b15698665ec4f97703. [Reporting paper]() Recorded evaluation configuration: {"backbone":"ministral-8b"}. This record does not assert independent reproduction. ### GCD Syntax Correctness Rate: 1. Result ID: 699a70b15698665ec4f9770d. [Reporting paper]() Recorded evaluation configuration: {"backbone":"qwen-2.5-7b"}. This record does not assert independent reproduction. Page 1 of 1. This is an index page, not the full collection.