# NL-to-FOL Syntax Correctness on LogicNLI (test) Source: [NL-to-FOL Syntax Correctness on LogicNLI (test)]() Collection index: [llms.txt]() Data updated: 2026-02-26T06:05:48.300Z Evaluation of model performance on determining the syntactic correctness of Natural Language representations converted into First-Order Logic (FOL) using the test split of the LogicNLI dataset. A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. | Scope | Recorded value | | --- | --- | | Task | NL-to-FOL Syntax Correctness | | Dataset (including any stated split) | LogicNLI (test) | | Metric | Syntax Correctness Rate | | Unit | PERCENT | | Direction | Higher is better | | Results loaded | 26 of 26 | | Coverage | All results currently returned for this benchmark | Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol. ## Metrics - [Syntax Correctness Rate](): Higher is better; unit: PERCENT. ## Recorded results | Method | Syntax Correctness Rate | Result date | Report paper | | --- | --- | --- | --- | | NL2LOGIC | 99 | Not recorded | 6993f6239ebc499c97409b3a | | NL2LOGIC | 99 | Not recorded | 6993f6239ebc499c97409b3a | | NL2LOGIC | 99 | Not recorded | 6993f6239ebc499c97409b3a | | NL2LOGIC | 99 | Not recorded | 6993f6239ebc499c97409b3a | | NL2LOGIC | 99 | Not recorded | 6993f6239ebc499c97409b3a | | NL2LOGIC | 99 | Not recorded | 6993f6239ebc499c97409b3a | | NL2LOGIC | 98 | Not recorded | 6993f6239ebc499c97409b3a | | NL2LOGIC | 98 | Not recorded | 6993f6239ebc499c97409b3a | | NL2LOGIC | 98 | Not recorded | 6993f6239ebc499c97409b3a | | NL2LOGIC | 98 | Not recorded | 6993f6239ebc499c97409b3a | | NL2LOGIC | 97 | Not recorded | 6993f6239ebc499c97409b3a | | NL2LOGIC | 94 | Not recorded | 6993f6239ebc499c97409b3a | | GCD | 93 | Not recorded | 6993f6239ebc499c97409b3a | | GCD | 91 | Not recorded | 6993f6239ebc499c97409b3a | | NL2LOGIC | 80 | Not recorded | 6993f6239ebc499c97409b3a | | GCD | 68 | Not recorded | 6993f6239ebc499c97409b3a | | GCD | 56 | Not recorded | 6993f6239ebc499c97409b3a | | GCD | 46 | Not recorded | 6993f6239ebc499c97409b3a | | GCD | 28 | Not recorded | 6993f6239ebc499c97409b3a | | GCD | 25 | Not recorded | 6993f6239ebc499c97409b3a | | GCD | 9 | Not recorded | 6993f6239ebc499c97409b3a | | GCD | 8 | Not recorded | 6993f6239ebc499c97409b3a | | GCD | 7 | Not recorded | 6993f6239ebc499c97409b3a | | GCD | 5 | Not recorded | 6993f6239ebc499c97409b3a | | GCD | 1 | Not recorded | 6993f6239ebc499c97409b3a | | GCD | 1 | Not recorded | 6993f6239ebc499c97409b3a | ### NL2LOGIC Syntax Correctness Rate: 99. Result ID: 699a70b15698665ec4f97714. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"backbone":"gemma-2-9b"}. This record does not assert independent reproduction. ### NL2LOGIC Syntax Correctness Rate: 99. Result ID: 699a70b15698665ec4f97716. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"backbone":"gemma-2-27b"}. This record does not assert independent reproduction. ### NL2LOGIC Syntax Correctness Rate: 99. Result ID: 699a70b15698665ec4f97724. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"backbone":"qwen-2.5-1.5b"}. This record does not assert independent reproduction. ### NL2LOGIC Syntax Correctness Rate: 99. Result ID: 699a70b15698665ec4f97726. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"backbone":"qwen-2.5-3b"}. This record does not assert independent reproduction. ### NL2LOGIC Syntax Correctness Rate: 99. Result ID: 699a70b15698665ec4f97728. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"backbone":"qwen-2.5-7b"}. This record does not assert independent reproduction. ### NL2LOGIC Syntax Correctness Rate: 99. Result ID: 699a70b15698665ec4f9772a. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"backbone":"qwen-2.5-14b"}. This record does not assert independent reproduction. ### NL2LOGIC Syntax Correctness Rate: 98. Result ID: 699a70b15698665ec4f9771a. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"backbone":"llama-3.2-3b"}. This record does not assert independent reproduction. ### NL2LOGIC Syntax Correctness Rate: 98. Result ID: 699a70b15698665ec4f9771c. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"backbone":"llama-3.1-8b"}. This record does not assert independent reproduction. ### NL2LOGIC Syntax Correctness Rate: 98. Result ID: 699a70b15698665ec4f9771e. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"backbone":"ministral-8b"}. This record does not assert independent reproduction. ### NL2LOGIC Syntax Correctness Rate: 98. Result ID: 699a70b15698665ec4f97720. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"backbone":"mistral-22b"}. This record does not assert independent reproduction. ### NL2LOGIC Syntax Correctness Rate: 97. Result ID: 699a70b15698665ec4f97718. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"backbone":"llama-3.2-1b"}. This record does not assert independent reproduction. ### NL2LOGIC Syntax Correctness Rate: 94. Result ID: 699a70b15698665ec4f97712. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"backbone":"gemma-2-2b"}. This record does not assert independent reproduction. ### GCD Syntax Correctness Rate: 93. Result ID: 699a70b15698665ec4f97715. [Reporting paper]() Recorded evaluation configuration: {"backbone":"gemma-2-27b"}. This record does not assert independent reproduction. ### GCD Syntax Correctness Rate: 91. Result ID: 699a70b15698665ec4f97713. [Reporting paper]() Recorded evaluation configuration: {"backbone":"gemma-2-9b"}. This record does not assert independent reproduction. ### NL2LOGIC Syntax Correctness Rate: 80. Result ID: 699a70b15698665ec4f97722. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"backbone":"qwen-2.5-0.5b"}. This record does not assert independent reproduction. ### GCD Syntax Correctness Rate: 68. Result ID: 699a70b15698665ec4f9771b. [Reporting paper]() Recorded evaluation configuration: {"backbone":"llama-3.1-8b"}. This record does not assert independent reproduction. ### GCD Syntax Correctness Rate: 56. Result ID: 699a70b15698665ec4f97729. [Reporting paper]() Recorded evaluation configuration: {"backbone":"qwen-2.5-14b"}. This record does not assert independent reproduction. ### GCD Syntax Correctness Rate: 46. Result ID: 699a70b15698665ec4f9771f. [Reporting paper]() Recorded evaluation configuration: {"backbone":"mistral-22b"}. This record does not assert independent reproduction. ### GCD Syntax Correctness Rate: 28. Result ID: 699a70b15698665ec4f97719. [Reporting paper]() Recorded evaluation configuration: {"backbone":"llama-3.2-3b"}. This record does not assert independent reproduction. ### GCD Syntax Correctness Rate: 25. Result ID: 699a70b15698665ec4f97711. [Reporting paper]() Recorded evaluation configuration: {"backbone":"gemma-2-2b"}. This record does not assert independent reproduction. ### GCD Syntax Correctness Rate: 9. Result ID: 699a70b15698665ec4f97723. [Reporting paper]() Recorded evaluation configuration: {"backbone":"qwen-2.5-1.5b"}. This record does not assert independent reproduction. ### GCD Syntax Correctness Rate: 8. Result ID: 699a70b15698665ec4f97717. [Reporting paper]() Recorded evaluation configuration: {"backbone":"llama-3.2-1b"}. This record does not assert independent reproduction. ### GCD Syntax Correctness Rate: 7. Result ID: 699a70b15698665ec4f9771d. [Reporting paper]() Recorded evaluation configuration: {"backbone":"ministral-8b"}. This record does not assert independent reproduction. ### GCD Syntax Correctness Rate: 5. Result ID: 699a70b15698665ec4f97721. [Reporting paper]() Recorded evaluation configuration: {"backbone":"qwen-2.5-0.5b"}. This record does not assert independent reproduction. ### GCD Syntax Correctness Rate: 1. Result ID: 699a70b15698665ec4f97725. [Reporting paper]() Recorded evaluation configuration: {"backbone":"qwen-2.5-3b"}. This record does not assert independent reproduction. ### GCD Syntax Correctness Rate: 1. Result ID: 699a70b15698665ec4f97727. [Reporting paper]() Recorded evaluation configuration: {"backbone":"qwen-2.5-7b"}. This record does not assert independent reproduction. Page 1 of 1. This is an index page, not the full collection.