# Natural Language Inference on LogicNLI Source: [Natural Language Inference on LogicNLI]() Collection index: [llms.txt]() Data updated: 2026-02-26T05:32:51.335Z Evaluation of Natural Language Inference performance on the LogicNLI dataset, measured by Accuracy. A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. | Scope | Recorded value | | --- | --- | | Task | Natural Language Inference | | Dataset (including any stated split) | LogicNLI | | Metric | Accuracy | | Unit | PERCENT | | Direction | Higher is better | | Results loaded | 26 of 26 | | Coverage | All results currently returned for this benchmark | Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol. ## Metrics - [Accuracy](): Higher is better; unit: PERCENT. ## Recorded results | Method | Accuracy | Result date | Report paper | | --- | --- | --- | --- | | gemma-2-2b | 41 | Not recorded | 6993f6239ebc499c97409b3a | | qwen-2.5-1.5b | 41 | Not recorded | 6993f6239ebc499c97409b3a | | ministral-8b | 38 | Not recorded | 6993f6239ebc499c97409b3a | | gemma-2-9b | 37 | Not recorded | 6993f6239ebc499c97409b3a | | gemma-2-27b | 37 | Not recorded | 6993f6239ebc499c97409b3a | | llama-3.2-3b | 37 | Not recorded | 6993f6239ebc499c97409b3a | | llama-3.1-8b | 37 | Not recorded | 6993f6239ebc499c97409b3a | | qwen-2.5-0.5b | 36 | Not recorded | 6993f6239ebc499c97409b3a | | qwen-2.5-3b | 35 | Not recorded | 6993f6239ebc499c97409b3a | | qwen-2.5-14b | 35 | Not recorded | 6993f6239ebc499c97409b3a | | mistral-22b | 34 | Not recorded | 6993f6239ebc499c97409b3a | | qwen-2.5-7b | 34 | Not recorded | 6993f6239ebc499c97409b3a | | llama-3.2-1b | 33 | Not recorded | 6993f6239ebc499c97409b3a | | qwen-2.5-14b | 27 | Not recorded | 6993f6239ebc499c97409b3a | | gemma-2-27b | 24 | Not recorded | 6993f6239ebc499c97409b3a | | gemma-2-9b | 23 | Not recorded | 6993f6239ebc499c97409b3a | | gemma-2-2b | 22 | Not recorded | 6993f6239ebc499c97409b3a | | llama-3.1-8b | 17 | Not recorded | 6993f6239ebc499c97409b3a | | llama-3.2-3b | 15 | Not recorded | 6993f6239ebc499c97409b3a | | mistral-22b | 14 | Not recorded | 6993f6239ebc499c97409b3a | | qwen-2.5-0.5b | 12 | Not recorded | 6993f6239ebc499c97409b3a | | llama-3.2-1b | 11 | Not recorded | 6993f6239ebc499c97409b3a | | ministral-8b | 4 | Not recorded | 6993f6239ebc499c97409b3a | | qwen-2.5-1.5b | 3 | Not recorded | 6993f6239ebc499c97409b3a | | qwen-2.5-3b | 1 | Not recorded | 6993f6239ebc499c97409b3a | | qwen-2.5-7b | 1 | Not recorded | 6993f6239ebc499c97409b3a | ### gemma-2-2b Accuracy: 41. Result ID: 699a70b15698665ec4f97746. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Decoding Method":"NL2LOGIC"}. This record does not assert independent reproduction. ### qwen-2.5-1.5b Accuracy: 41. Result ID: 699a70b15698665ec4f97758. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Decoding Method":"NL2LOGIC"}. This record does not assert independent reproduction. ### ministral-8b Accuracy: 38. Result ID: 699a70b15698665ec4f97752. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Decoding Method":"NL2LOGIC"}. This record does not assert independent reproduction. ### gemma-2-9b Accuracy: 37. Result ID: 699a70b15698665ec4f97748. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Decoding Method":"NL2LOGIC"}. This record does not assert independent reproduction. ### gemma-2-27b Accuracy: 37. Result ID: 699a70b15698665ec4f9774a. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Decoding Method":"NL2LOGIC"}. This record does not assert independent reproduction. ### llama-3.2-3b Accuracy: 37. Result ID: 699a70b15698665ec4f9774e. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Decoding Method":"NL2LOGIC"}. This record does not assert independent reproduction. ### llama-3.1-8b Accuracy: 37. Result ID: 699a70b15698665ec4f97750. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Decoding Method":"NL2LOGIC"}. This record does not assert independent reproduction. ### qwen-2.5-0.5b Accuracy: 36. Result ID: 699a70b15698665ec4f97756. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Decoding Method":"NL2LOGIC"}. This record does not assert independent reproduction. ### qwen-2.5-3b Accuracy: 35. Result ID: 699a70b15698665ec4f9775a. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Decoding Method":"NL2LOGIC"}. This record does not assert independent reproduction. ### qwen-2.5-14b Accuracy: 35. Result ID: 699a70b15698665ec4f9775e. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Decoding Method":"NL2LOGIC"}. This record does not assert independent reproduction. ### mistral-22b Accuracy: 34. Result ID: 699a70b15698665ec4f97754. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Decoding Method":"NL2LOGIC"}. This record does not assert independent reproduction. ### qwen-2.5-7b Accuracy: 34. Result ID: 699a70b15698665ec4f9775c. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Decoding Method":"NL2LOGIC"}. This record does not assert independent reproduction. ### llama-3.2-1b Accuracy: 33. Result ID: 699a70b15698665ec4f9774c. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Decoding Method":"NL2LOGIC"}. This record does not assert independent reproduction. ### qwen-2.5-14b Accuracy: 27. Result ID: 699a70b15698665ec4f9775d. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Decoding Method":"GCD"}. This record does not assert independent reproduction. ### gemma-2-27b Accuracy: 24. Result ID: 699a70b15698665ec4f97749. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Decoding Method":"GCD"}. This record does not assert independent reproduction. ### gemma-2-9b Accuracy: 23. Result ID: 699a70b15698665ec4f97747. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Decoding Method":"GCD"}. This record does not assert independent reproduction. ### gemma-2-2b Accuracy: 22. Result ID: 699a70b15698665ec4f97745. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Decoding Method":"GCD"}. This record does not assert independent reproduction. ### llama-3.1-8b Accuracy: 17. Result ID: 699a70b15698665ec4f9774f. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Decoding Method":"GCD"}. This record does not assert independent reproduction. ### llama-3.2-3b Accuracy: 15. Result ID: 699a70b15698665ec4f9774d. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Decoding Method":"GCD"}. This record does not assert independent reproduction. ### mistral-22b Accuracy: 14. Result ID: 699a70b15698665ec4f97753. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Decoding Method":"GCD"}. This record does not assert independent reproduction. ### qwen-2.5-0.5b Accuracy: 12. Result ID: 699a70b15698665ec4f97755. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Decoding Method":"GCD"}. This record does not assert independent reproduction. ### llama-3.2-1b Accuracy: 11. Result ID: 699a70b15698665ec4f9774b. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Decoding Method":"GCD"}. This record does not assert independent reproduction. ### ministral-8b Accuracy: 4. Result ID: 699a70b15698665ec4f97751. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Decoding Method":"GCD"}. This record does not assert independent reproduction. ### qwen-2.5-1.5b Accuracy: 3. Result ID: 699a70b15698665ec4f97757. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Decoding Method":"GCD"}. This record does not assert independent reproduction. ### qwen-2.5-3b Accuracy: 1. Result ID: 699a70b15698665ec4f97759. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Decoding Method":"GCD"}. This record does not assert independent reproduction. ### qwen-2.5-7b Accuracy: 1. Result ID: 699a70b15698665ec4f9775b. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Decoding Method":"GCD"}. This record does not assert independent reproduction. Page 1 of 1. This is an index page, not the full collection.