# Natural Language Inference on NLI

Source: [Natural Language Inference on NLI](<https://www.sota2.com/research/sota/natural-language-inference-on-nli>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2026-04-03T06:57:44.435Z

Evaluation of Natural Language Inference performance on the NLI dataset, measured by Accuracy.

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

| Scope | Recorded value |
| --- | --- |
| Task | Natural Language Inference |
| Dataset (including any stated split) | NLI |
| Metric | Accuracy |
| Unit | PERCENT |
| Direction | Higher is better |
| Results loaded | 14 of 14 |
| Coverage | All results currently returned for this benchmark |

Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol.

## Metrics

- [Accuracy](<https://www.sota2.com/research/sota/natural-language-inference-on-nli.md?metric=Accuracy>): Higher is better; unit: PERCENT.

- [ECE](<https://www.sota2.com/research/sota/natural-language-inference-on-nli.md?metric=ECE>): Lower is better; unit: PERCENT.

## Recorded results

| Method | Accuracy | Result date | Report paper |
| --- | --- | --- | --- |
| EA | 91.2 | Not recorded | 6948d095423ea4226572a07b |
| EA | 91.14 | Not recorded | 6948d095423ea4226572a07b |
| EA | 90.92 | Not recorded | 6948d095423ea4226572a07b |
| GLM-4-Plus | 90.56 | Not recorded | 6948d095423ea4226572a07b |
| EA | 90.18 | Not recorded | 6948d095423ea4226572a07b |
| Qwen-Max | 90.1 | Not recorded | 6948d095423ea4226572a07b |
| Doubao-1.5-pro-32k | 89.98 | Not recorded | 6948d095423ea4226572a07b |
| Hunyuan-Turbos | 88.62 | Not recorded | 6948d095423ea4226572a07b |
| ABMLL-MetaICL | 85.2 | Not recorded | 68fa7ab71e209131cea84c0e |
| MetaICL | 83.8 | Not recorded | 68fa7ab71e209131cea84c0e |
| Reptile-MetaICL | 83.6 | Not recorded | 68fa7ab71e209131cea84c0e |
| MetaICL | 83.3 | Not recorded | 68fa7ab71e209131cea84c0e |
| Reptile-MetaICL | 83 | Not recorded | 68fa7ab71e209131cea84c0e |
| ABMLL-MetaICL | 82.6 | Not recorded | 68fa7ab71e209131cea84c0e |

### EA

Accuracy: 91.2. Result ID: 69994e905698665ec4b8e1f0.

[Reporting paper](<https://www.sota2.com/research/paper/6948d095423ea4226572a07b.md>)

[Method origin paper](<https://www.sota2.com/research/paper/6948d095423ea4226572a07b.md>)

Recorded evaluation configuration: {"Base Model":"GLM-4-Plus"}. This record does not assert independent reproduction.

### EA

Accuracy: 91.14. Result ID: 69994e905698665ec4b8e1f2.

[Reporting paper](<https://www.sota2.com/research/paper/6948d095423ea4226572a07b.md>)

[Method origin paper](<https://www.sota2.com/research/paper/6948d095423ea4226572a07b.md>)

Recorded evaluation configuration: {"Base Model":"Doubao-1.5-pro-32k"}. This record does not assert independent reproduction.

### EA

Accuracy: 90.92. Result ID: 69994e905698665ec4b8e1f4.

[Reporting paper](<https://www.sota2.com/research/paper/6948d095423ea4226572a07b.md>)

[Method origin paper](<https://www.sota2.com/research/paper/6948d095423ea4226572a07b.md>)

Recorded evaluation configuration: {"Base Model":"Qwen-Max"}. This record does not assert independent reproduction.

### GLM-4-Plus

Accuracy: 90.56. Result ID: 69994e905698665ec4b8e1ef.

[Reporting paper](<https://www.sota2.com/research/paper/6948d095423ea4226572a07b.md>)

Recorded evaluation configuration: Not recorded. This record does not assert independent reproduction.

### EA

Accuracy: 90.18. Result ID: 69994e905698665ec4b8e1f6.

[Reporting paper](<https://www.sota2.com/research/paper/6948d095423ea4226572a07b.md>)

[Method origin paper](<https://www.sota2.com/research/paper/6948d095423ea4226572a07b.md>)

Recorded evaluation configuration: {"Base Model":"Hunyuan-Turbos"}. This record does not assert independent reproduction.

### Qwen-Max

Accuracy: 90.1. Result ID: 69994e905698665ec4b8e1f3.

[Reporting paper](<https://www.sota2.com/research/paper/6948d095423ea4226572a07b.md>)

Recorded evaluation configuration: Not recorded. This record does not assert independent reproduction.

### Doubao-1.5-pro-32k

Accuracy: 89.98. Result ID: 69994e905698665ec4b8e1f1.

[Reporting paper](<https://www.sota2.com/research/paper/6948d095423ea4226572a07b.md>)

Recorded evaluation configuration: Not recorded. This record does not assert independent reproduction.

### Hunyuan-Turbos

Accuracy: 88.62. Result ID: 69994e905698665ec4b8e1f5.

[Reporting paper](<https://www.sota2.com/research/paper/6948d095423ea4226572a07b.md>)

Recorded evaluation configuration: Not recorded. This record does not assert independent reproduction.

### ABMLL-MetaICL

Accuracy: 85.2. Result ID: 69cf64d847f34ce924b8b021.

[Reporting paper](<https://www.sota2.com/research/paper/68fa7ab71e209131cea84c0e.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa7ab71e209131cea84c0e.md>)

Recorded evaluation configuration: {"Model":"LLAMA3","ICL setting":"Yes"}. This record does not assert independent reproduction.

### MetaICL

Accuracy: 83.8. Result ID: 69cf64d847f34ce924b8b022.

[Reporting paper](<https://www.sota2.com/research/paper/68fa7ab71e209131cea84c0e.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa6c7c1e209131cea34419.md>)

Recorded evaluation configuration: {"Model":"QWEN2","ICL setting":"Yes"}. This record does not assert independent reproduction.

### Reptile-MetaICL

Accuracy: 83.6. Result ID: 69cf64d847f34ce924b8b020.

[Reporting paper](<https://www.sota2.com/research/paper/68fa7ab71e209131cea84c0e.md>)

Recorded evaluation configuration: {"Model":"LLAMA3","ICL setting":"Yes"}. This record does not assert independent reproduction.

### MetaICL

Accuracy: 83.3. Result ID: 69cf64d847f34ce924b8b01f.

[Reporting paper](<https://www.sota2.com/research/paper/68fa7ab71e209131cea84c0e.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa6c7c1e209131cea34419.md>)

Recorded evaluation configuration: {"Model":"LLAMA3","ICL setting":"Yes"}. This record does not assert independent reproduction.

### Reptile-MetaICL

Accuracy: 83. Result ID: 69cf64d847f34ce924b8b023.

[Reporting paper](<https://www.sota2.com/research/paper/68fa7ab71e209131cea84c0e.md>)

Recorded evaluation configuration: {"Model":"QWEN2","ICL setting":"Yes"}. This record does not assert independent reproduction.

### ABMLL-MetaICL

Accuracy: 82.6. Result ID: 69cf64d847f34ce924b8b024.

[Reporting paper](<https://www.sota2.com/research/paper/68fa7ab71e209131cea84c0e.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa7ab71e209131cea84c0e.md>)

Recorded evaluation configuration: {"Model":"QWEN2","ICL setting":"Yes"}. This record does not assert independent reproduction.

Page 1 of 1. This is an index page, not the full collection.
