Textual Entailment on SICK (test)
90.3AccuracyNeuralLog (full system)
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| NeuralLog (full system)System Category=Our System2021.05 | 90.3 | 88 | 87.6 | — | |
| Yin and Schütze (2017)System Category=ML/DL-based systems2021.05 | 87.1 | — | — | — | |
| JMTvariant=DE2016.11 | 86.8 | — | — | — | |
| BERT (base, uncased)System Category=ML/DL-based systems2021.05 | 86.7 | 86.8 | 85.4 | — | |
| Kalouli et al. (2020)System Category=Hybrid System2021.05 | 86.5 | — | — | — | |
| InferSentEvaluation Protocol=SentEval2018.08 | 86.3 | — | — | — | |
| JMTvariant=all2016.11 | 86.2 | — | — | — | |
| Yin et al.year=20162016.11 | 86.2 | — | — | — | |
| Hu et al. (2020)-BERTSystem Category=Hybrid System2021.05 | 85.4 | 83.2 | 85.5 | — | |
| HBMPEmbedding Dimension=600D, Evaluation Protocol=SentEval2018.08 | 85.3 | — | — | — | |
| Beltagy et al. (2016)System Category=ML/DL-based systems2021.05 | 85.1 | — | — | — | |
| HBMPEmbedding Dimension=1200D, Evaluation Protocol=SentEval2018.08 | 84.7 | — | — | — | |
| Lai and Hockenmaieryear=20142016.11 | 84.6 | — | — | — | |
| Abzianidze (2020)System Category=Logic-based systems2021.05 | 84.4 | 94.3 | 67.9 | — | |
| Yanaka et al. (2018)System Category=Logic-based systems2021.05 | 84.3 | 84.2 | 77.3 | — | |
| Martínez-Gómez et al. (2017)System Category=Logic-based systems2021.05 | 83.1 | 97 | 63.6 | — | |
| Abzianidze (2017)System Category=Logic-based systems2021.05 | 81.4 | 98 | 58.1 | — | |
| SkipThoughtEvaluation Protocol=SentEval2018.08 | 79.5 | — | — | — | |
| Hu et al. (2020)System Category=Logic-based systems2021.05 | 77.2 | 83.8 | 70.7 | — | |
| NeuralLog (- Monotonicity)System Category=Our System, Ablation=Remove monotonicity-based inference modules2021.05 | 74.7 | 74.5 | 75.1 | — | |
| NeuralLog (- ALBERT-SV)System Category=Our System, Ablation=Remove syntactic variation module2021.05 | 71.4 | 68.9 | 79.3 | — | |
| RoBERTa-large (Context Calibration)Model=RoBERTa-large, Setting=CC, Evaluation protocol=Zero-shot prompting2023.05 | — | — | — | 39.7 | |
| RoBERTa-large (Domain-context Calibration)Model=RoBERTa-large, Setting=DC, Evaluation protocol=Zero-shot prompting2023.05 | — | — | — | 54.2 | |
| RoBERTa-large (Original)Model=RoBERTa-large, Setting=Original, Evaluation protocol=Zero-shot prompting2023.05 | — | — | — | 38.2 |