# Natural Language Inference on MultiNLI matched (in-domain) RepEval 2017 (test) Source: [Natural Language Inference on MultiNLI matched (in-domain) RepEval 2017 (test)]() Collection index: [llms.txt]() Data updated: 2026-02-26T05:27:15.891Z Evaluation of Natural Language Inference performance on the matched (in-domain) test split from the MultiNLI RepEval 2017 benchmark, typically measured by accuracy. A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. | Scope | Recorded value | | --- | --- | | Task | Natural Language Inference | | Dataset (including any stated split) | MultiNLI matched (in-domain) RepEval 2017 (test) | | Metric | Accuracy | | Unit | PERCENT | | Direction | Higher is better | | Results loaded | 18 of 18 | | Coverage | All results currently returned for this benchmark | Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol. ## Metrics - [Accuracy](): Higher is better; unit: PERCENT. ## Recorded results | Method | Accuracy | Result date | Report paper | | --- | --- | --- | --- | | Our ESIM | 76.8 | Not recorded | 68fa638b1e209131ce9ff741 | | Ensembled | 74.9 | Not recorded | 68fa638b1e209131ce9ff741 | | YixinNie-UNC-NLP | 74.5 | Not recorded | 68fa638b1e209131ce9ff741 | | Single (Output Gate) | 73.7 | Not recorded | 68fa638b1e209131ce9ff741 | | Single | 73.5 | Not recorded | 68fa638b1e209131ce9ff741 | | Single (Input Gate) | 73.5 | Not recorded | 68fa638b1e209131ce9ff741 | | Single (Forget Gate) | 72.9 | Not recorded | 68fa638b1e209131ce9ff741 | | Single - CharCNN | 72.9 | Not recorded | 68fa638b1e209131ce9ff741 | | Single - Gated-Att | 72.8 | Not recorded | 68fa638b1e209131ce9ff741 | | ESIM | 72.3 | Not recorded | 68fa638b1e209131ce9ff741 | | Rivercorners | 72.2 | Not recorded | 68fa638b1e209131ce9ff741 | | Rivercorners | 72.1 | Not recorded | 68fa638b1e209131ce9ff741 | | LCT-MALTA | 70.7 | Not recorded | 68fa638b1e209131ce9ff741 | | Single - AbsDiff/Product | 69.7 | Not recorded | 68fa638b1e209131ce9ff741 | | TALP-UPC | 67.9 | Not recorded | 68fa638b1e209131ce9ff741 | | BiLSTM | 66.9 | Not recorded | 68fa638b1e209131ce9ff741 | | Single - Word Embedding | 65.6 | Not recorded | 68fa638b1e209131ce9ff741 | | CBOW | 64.8 | Not recorded | 68fa638b1e209131ce9ff741 | ### Our ESIM Accuracy: 76.8. Result ID: 698bfb81f729bc7f68e73447. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"implementation":"authors"}. This record does not assert independent reproduction. ### Ensembled Accuracy: 74.9. Result ID: 698bfb81f729bc7f68e73449. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"ensemble":true,"RepEval 2017 participant":true}. This record does not assert independent reproduction. ### YixinNie-UNC-NLP Accuracy: 74.5. Result ID: 698bfb81f729bc7f68e73446. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"RepEval 2017 participant":true}. This record does not assert independent reproduction. ### Single (Output Gate) Accuracy: 73.7. Result ID: 698bfb81f729bc7f68e7344c. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"gate\_type":"output"}. This record does not assert independent reproduction. ### Single Accuracy: 73.5. Result ID: 698bfb81f729bc7f68e73448. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"ensemble":false,"RepEval 2017 participant":true}. This record does not assert independent reproduction. ### Single (Input Gate) Accuracy: 73.5. Result ID: 698bfb81f729bc7f68e7344a. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"gate\_type":"input","RepEval 2017 participant":true}. This record does not assert independent reproduction. ### Single (Forget Gate) Accuracy: 72.9. Result ID: 698bfb81f729bc7f68e7344b. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"gate\_type":"forget"}. This record does not assert independent reproduction. ### Single - CharCNN Accuracy: 72.9. Result ID: 698bfb81f729bc7f68e7344e. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"character\_composition":"removed"}. This record does not assert independent reproduction. ### Single - Gated-Att Accuracy: 72.8. Result ID: 698bfb81f729bc7f68e7344d. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"gated\_attention":"removed"}. This record does not assert independent reproduction. ### ESIM Accuracy: 72.3. Result ID: 698bfb81f729bc7f68e73441. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: Not recorded. This record does not assert independent reproduction. ### Rivercorners Accuracy: 72.2. Result ID: 698bfb81f729bc7f68e73445. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"RepEval 2017 participant":true,"ensemble":true}. This record does not assert independent reproduction. ### Rivercorners Accuracy: 72.1. Result ID: 698bfb81f729bc7f68e73444. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"RepEval 2017 participant":true,"ensemble":false}. This record does not assert independent reproduction. ### LCT-MALTA Accuracy: 70.7. Result ID: 698bfb81f729bc7f68e73443. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"RepEval 2017 participant":true}. This record does not assert independent reproduction. ### Single - AbsDiff/Product Accuracy: 69.7. Result ID: 698bfb81f729bc7f68e73450. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"elementwise\_vector\_comparison":"removed"}. This record does not assert independent reproduction. ### TALP-UPC Accuracy: 67.9. Result ID: 698bfb81f729bc7f68e73442. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"RepEval 2017 participant":true}. This record does not assert independent reproduction. ### BiLSTM Accuracy: 66.9. Result ID: 698bfb81f729bc7f68e73440. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: Not recorded. This record does not assert independent reproduction. ### Single - Word Embedding Accuracy: 65.6. Result ID: 698bfb81f729bc7f68e7344f. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"word\_level\_embedding":"removed"}. This record does not assert independent reproduction. ### CBOW Accuracy: 64.8. Result ID: 698bfb81f729bc7f68e7343f. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: Not recorded. This record does not assert independent reproduction. Page 1 of 1. This is an index page, not the full collection.