# Natural Language Inference on MultiNLI mismatched (cross-domain) RepEval 2017 (test) Source: [Natural Language Inference on MultiNLI mismatched (cross-domain) RepEval 2017 (test)]() Collection index: [llms.txt]() Data updated: 2026-02-26T05:27:22.445Z Evaluation of Natural Language Inference performance on the mismatched (cross-domain) test split of the MultiNLI dataset, typically measured by accuracy. A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. | Scope | Recorded value | | --- | --- | | Task | Natural Language Inference | | Dataset (including any stated split) | MultiNLI mismatched (cross-domain) RepEval 2017 (test) | | Metric | Accuracy | | Unit | PERCENT | | Direction | Higher is better | | Results loaded | 25 of 25 | | Coverage | All results currently returned for this benchmark | Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol. ## Metrics - [Accuracy](): Higher is better; unit: PERCENT. ## Recorded results | Method | Accuracy | Result date | Report paper | | --- | --- | --- | --- | | Our ESIM | 75.8 | Not recorded | 68fa638b1e209131ce9ff741 | | Ensembled | 74.9 | Not recorded | 68fa638b1e209131ce9ff741 | | Single | 73.6 | Not recorded | 68fa638b1e209131ce9ff741 | | Single (Input Gate) | 73.6 | Not recorded | 68fa638b1e209131ce9ff741 | | Single - Gated-Att | 73.6 | Not recorded | 68fa638b1e209131ce9ff741 | | Shortcut-Stacked Encoder | 73.6 | Not recorded | 68fa63d11e209131cea00ec6 | | YixinNie-UNC-NLP | 73.5 | Not recorded | 68fa638b1e209131ce9ff741 | | Single - CharCNN | 73.5 | Not recorded | 68fa638b1e209131ce9ff741 | | Single (Output Gate) | 73.4 | Not recorded | 68fa638b1e209131ce9ff741 | | Single (Forget Gate) | 73.1 | Not recorded | 68fa638b1e209131ce9ff741 | | Rivercorners | 72.8 | Not recorded | 68fa638b1e209131ce9ff741 | | ESIM | 72.1 | Not recorded | 68fa638b1e209131ce9ff741 | | Rivercorners | 72.1 | Not recorded | 68fa638b1e209131ce9ff741 | | DiSAN | 71.402 | Not recorded | 68fa63cf1e209131cea00caa | | biLSTM-Max Encoder | 71.2 | Not recorded | 68fa63d11e209131cea00ec6 | | LCT-MALTA | 70.8 | Not recorded | 68fa638b1e209131ce9ff741 | | Single - AbsDiff/Product | 69.2 | Not recorded | 68fa638b1e209131ce9ff741 | | TALP-UPC | 68.2 | Not recorded | 68fa638b1e209131ce9ff741 | | Bi-LSTM | 67.248 | Not recorded | 68fa63cf1e209131cea00caa | | biLSTM Encoder | 67.1 | Not recorded | 68fa63d11e209131cea00ec6 | | BiLSTM | 66.9 | Not recorded | 68fa638b1e209131ce9ff741 | | Single - Word Embedding | 66 | Not recorded | 68fa638b1e209131ce9ff741 | | CBOW | 64.759 | Not recorded | 68fa63cf1e209131cea00caa | | CBOW | 64.6 | Not recorded | 68fa63d11e209131cea00ec6 | | CBOW | 64.5 | Not recorded | 68fa638b1e209131ce9ff741 | ### Our ESIM Accuracy: 75.8. Result ID: 698bfb81f729bc7f68e73459. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"implementation":"authors"}. This record does not assert independent reproduction. ### Ensembled Accuracy: 74.9. Result ID: 698bfb81f729bc7f68e7345b. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"ensemble":true,"RepEval 2017 participant":true}. This record does not assert independent reproduction. ### Single Accuracy: 73.6. Result ID: 698bfb81f729bc7f68e7345a. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"ensemble":false,"RepEval 2017 participant":true}. This record does not assert independent reproduction. ### Single (Input Gate) Accuracy: 73.6. Result ID: 698bfb81f729bc7f68e7345c. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"gate\_type":"input","RepEval 2017 participant":true}. This record does not assert independent reproduction. ### Single - Gated-Att Accuracy: 73.6. Result ID: 698bfb81f729bc7f68e7345f. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"gated\_attention":"removed"}. This record does not assert independent reproduction. ### Shortcut-Stacked Encoder Accuracy: 73.6. Result ID: 698c798d37894ef7d9cea7c2. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: Not recorded. This record does not assert independent reproduction. ### YixinNie-UNC-NLP Accuracy: 73.5. Result ID: 698bfb81f729bc7f68e73458. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"RepEval 2017 participant":true}. This record does not assert independent reproduction. ### Single - CharCNN Accuracy: 73.5. Result ID: 698bfb81f729bc7f68e73460. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"character\_composition":"removed"}. This record does not assert independent reproduction. ### Single (Output Gate) Accuracy: 73.4. Result ID: 698bfb81f729bc7f68e7345e. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"gate\_type":"output"}. This record does not assert independent reproduction. ### Single (Forget Gate) Accuracy: 73.1. Result ID: 698bfb81f729bc7f68e7345d. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"gate\_type":"forget"}. This record does not assert independent reproduction. ### Rivercorners Accuracy: 72.8. Result ID: 698bfb81f729bc7f68e73457. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"RepEval 2017 participant":true,"ensemble":true}. This record does not assert independent reproduction. ### ESIM Accuracy: 72.1. Result ID: 698bfb81f729bc7f68e73453. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: Not recorded. This record does not assert independent reproduction. ### Rivercorners Accuracy: 72.1. Result ID: 698bfb81f729bc7f68e73456. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"RepEval 2017 participant":true,"ensemble":false}. This record does not assert independent reproduction. ### DiSAN Accuracy: 71.402. Result ID: 698c793137894ef7d9cea5e4. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: Not recorded. This record does not assert independent reproduction. ### biLSTM-Max Encoder Accuracy: 71.2. Result ID: 698c798d37894ef7d9cea7c1. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"reimplementation":true}. This record does not assert independent reproduction. ### LCT-MALTA Accuracy: 70.8. Result ID: 698bfb81f729bc7f68e73455. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"RepEval 2017 participant":true}. This record does not assert independent reproduction. ### Single - AbsDiff/Product Accuracy: 69.2. Result ID: 698bfb81f729bc7f68e73462. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"elementwise\_vector\_comparison":"removed"}. This record does not assert independent reproduction. ### TALP-UPC Accuracy: 68.2. Result ID: 698bfb81f729bc7f68e73454. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"RepEval 2017 participant":true}. This record does not assert independent reproduction. ### Bi-LSTM Accuracy: 67.248. Result ID: 698c793137894ef7d9cea5e3. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: Not recorded. This record does not assert independent reproduction. ### biLSTM Encoder Accuracy: 67.1. Result ID: 698c798d37894ef7d9cea7c0. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: Not recorded. This record does not assert independent reproduction. ### BiLSTM Accuracy: 66.9. Result ID: 698bfb81f729bc7f68e73452. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: Not recorded. This record does not assert independent reproduction. ### Single - Word Embedding Accuracy: 66. Result ID: 698bfb81f729bc7f68e73461. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"word\_level\_embedding":"removed"}. This record does not assert independent reproduction. ### CBOW Accuracy: 64.759. Result ID: 698c793137894ef7d9cea5e2. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: Not recorded. This record does not assert independent reproduction. ### CBOW Accuracy: 64.6. Result ID: 698c798d37894ef7d9cea7bf. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: Not recorded. This record does not assert independent reproduction. ### CBOW Accuracy: 64.5. Result ID: 698bfb81f729bc7f68e73451. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: Not recorded. This record does not assert independent reproduction. Page 1 of 1. This is an index page, not the full collection.