# Natural Language Inference on ANLI Source: [Natural Language Inference on ANLI]() Collection index: [llms.txt]() Data updated: 2026-02-26T05:22:33.202Z Evaluation of Natural Language Inference performance on the Adversarial Natural Language Inference (ANLI) dataset, covering results across Round 1, Round 2, and Round 3. A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. | Scope | Recorded value | | --- | --- | | Task | Natural Language Inference | | Dataset (including any stated split) | ANLI | | Metric | ANLI R1 Accuracy | | Unit | PERCENT | | Direction | Higher is better | | Results loaded | 27 of 27 | | Coverage | All results currently returned for this benchmark | Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol. ## Metrics - [ANLI R1 Accuracy](): Higher is better; unit: PERCENT. - [ANLI R1 Std Dev](): Lower is better; unit: SCALAR. - [ANLI R2 Accuracy](): Higher is better; unit: PERCENT. - [ANLI R2 Std Dev](): Lower is better; unit: SCALAR. - [ANLI R3 Accuracy](): Higher is better; unit: PERCENT. - [ANLI R3 Std Dev](): Lower is better; unit: SCALAR. ## Recorded results | Method | ANLI R1 Accuracy | Result date | Report paper | | --- | --- | --- | --- | | PaLM-2 L | 73.1 | Not recorded | 68fa715b1e209131cea50cd0 | | Falcon-180B | 60.5 | Not recorded | 68fa715b1e209131cea50cd0 | | PaLM-2 M | 58.1 | Not recorded | 68fa715b1e209131cea50cd0 | | PaLM-2 S | 53.1 | Not recorded | 68fa715b1e209131cea50cd0 | | PaLM | 52.6 | Not recorded | 68fa715b1e209131cea50cd0 | | KCReg | 48.2 | Not recorded | 68fa75841e209131cea6854b | | KCReg | 47.1 | Not recorded | 68fa75841e209131cea6854b | | Base | 46.1 | Not recorded | 68fa75841e209131cea6854b | | KCReg | 45.6 | Not recorded | 68fa75841e209131cea6854b | | ALUM | 45.2 | Not recorded | 68fa75841e209131cea6854b | | T0-11B | 44.7 | Not recorded | 68fa6ee81e209131cea42cc4 | | Base | 44.5 | Not recorded | 68fa75841e209131cea6854b | | ALUM | 44.5 | Not recorded | 68fa75841e209131cea6854b | | ALUM | 43.8 | Not recorded | 68fa75841e209131cea6854b | | Base | 42.7 | Not recorded | 68fa75841e209131cea6854b | | KiC-Large | 36.3 | Not recorded | 68fa6ee81e209131cea42cc4 | | GPT-J | 34 | Not recorded | 68fa6ee81e209131cea42cc4 | | BERT | 33.8 | Not recorded | 68fa6ee81e209131cea42cc4 | | T0-3B | 33.7 | Not recorded | 68fa6ee81e209131cea42cc4 | | GPT-Neo | 33.5 | Not recorded | 68fa6ee81e209131cea42cc4 | | RoBERTa | 33.4 | Not recorded | 68fa6ee81e209131cea42cc4 | | GPT-NeoX | 33.4 | Not recorded | 68fa6ee81e209131cea42cc4 | | OPT | 33.3 | Not recorded | 68fa6ee81e209131cea42cc4 | | KiC-Small | 33.3 | Not recorded | 68fa6ee81e209131cea42cc4 | | T0-Base | 32.2 | Not recorded | 68fa6ee81e209131cea42cc4 | | T0-Large | 30.5 | Not recorded | 68fa6ee81e209131cea42cc4 | | KiC-Base | 28.4 | Not recorded | 68fa6ee81e209131cea42cc4 | ### PaLM-2 L ANLI R1 Accuracy: 73.1. Result ID: 6991c8e93a6c3148490a913d. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"protocol":"1-shot"}. This record does not assert independent reproduction. ### Falcon-180B ANLI R1 Accuracy: 60.5. Result ID: 6991c8e93a6c3148490a913e. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"protocol":"1-shot"}. This record does not assert independent reproduction. ### PaLM-2 M ANLI R1 Accuracy: 58.1. Result ID: 6991c8e93a6c3148490a913c. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"protocol":"1-shot"}. This record does not assert independent reproduction. ### PaLM-2 S ANLI R1 Accuracy: 53.1. Result ID: 6991c8e93a6c3148490a913b. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"protocol":"1-shot"}. This record does not assert independent reproduction. ### PaLM ANLI R1 Accuracy: 52.6. Result ID: 6991c8e93a6c3148490a913a. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"protocol":"1-shot"}. This record does not assert independent reproduction. ### KCReg ANLI R1 Accuracy: 48.2. Result ID: 6993ccba3a6c31484918a0b8. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Architecture":"T5 \[57\]","Parameters":"~220M"}. This record does not assert independent reproduction. ### KCReg ANLI R1 Accuracy: 47.1. Result ID: 6993ccba3a6c31484918a0b5. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Architecture":"GPT2 \[56\]","Parameters":"~1.5B"}. This record does not assert independent reproduction. ### Base ANLI R1 Accuracy: 46.1. Result ID: 6993ccba3a6c31484918a0b6. [Reporting paper]() Recorded evaluation configuration: {"Architecture":"T5 \[57\]","Parameters":"~220M"}. This record does not assert independent reproduction. ### KCReg ANLI R1 Accuracy: 45.6. Result ID: 6993ccba3a6c31484918a0b2. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Architecture":"BERT \[16\]","Parameters":"~110M"}. This record does not assert independent reproduction. ### ALUM ANLI R1 Accuracy: 45.2. Result ID: 6993ccba3a6c31484918a0b1. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Architecture":"BERT \[16\]","Parameters":"~110M"}. This record does not assert independent reproduction. ### T0-11B ANLI R1 Accuracy: 44.7. Result ID: 699083733a6c314849011aa9. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Params":"11B","Evaluation":"Zero-shot"}. This record does not assert independent reproduction. ### Base ANLI R1 Accuracy: 44.5. Result ID: 6993ccba3a6c31484918a0b0. [Reporting paper]() Recorded evaluation configuration: {"Architecture":"BERT \[16\]","Parameters":"~110M"}. This record does not assert independent reproduction. ### ALUM ANLI R1 Accuracy: 44.5. Result ID: 6993ccba3a6c31484918a0b7. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Architecture":"T5 \[57\]","Parameters":"~220M"}. This record does not assert independent reproduction. ### ALUM ANLI R1 Accuracy: 43.8. Result ID: 6993ccba3a6c31484918a0b4. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Architecture":"GPT2 \[56\]","Parameters":"~1.5B"}. This record does not assert independent reproduction. ### Base ANLI R1 Accuracy: 42.7. Result ID: 6993ccba3a6c31484918a0b3. [Reporting paper]() Recorded evaluation configuration: {"Architecture":"GPT2 \[56\]","Parameters":"~1.5B"}. This record does not assert independent reproduction. ### KiC-Large ANLI R1 Accuracy: 36.3. Result ID: 699083733a6c314849011aac. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Params":"0.77B","Evaluation":"Zero-shot"}. This record does not assert independent reproduction. ### GPT-J ANLI R1 Accuracy: 34. Result ID: 699083733a6c314849011aa3. [Reporting paper]() Recorded evaluation configuration: {"Params":"6B","Evaluation":"Zero-shot"}. This record does not assert independent reproduction. ### BERT ANLI R1 Accuracy: 33.8. Result ID: 699083733a6c314849011aa0. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Params":"0.34B","Evaluation":"Zero-shot"}. This record does not assert independent reproduction. ### T0-3B ANLI R1 Accuracy: 33.7. Result ID: 699083733a6c314849011aa8. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Params":"3B","Evaluation":"Zero-shot"}. This record does not assert independent reproduction. ### GPT-Neo ANLI R1 Accuracy: 33.5. Result ID: 699083733a6c314849011aa2. [Reporting paper]() Recorded evaluation configuration: {"Params":"2.7B","Evaluation":"Zero-shot"}. This record does not assert independent reproduction. ### RoBERTa ANLI R1 Accuracy: 33.4. Result ID: 699083733a6c314849011aa1. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Params":"0.35B","Evaluation":"Zero-shot"}. This record does not assert independent reproduction. ### GPT-NeoX ANLI R1 Accuracy: 33.4. Result ID: 699083733a6c314849011aa5. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Params":"20B","Evaluation":"Zero-shot"}. This record does not assert independent reproduction. ### OPT ANLI R1 Accuracy: 33.3. Result ID: 699083733a6c314849011aa4. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Params":"30B","Evaluation":"Zero-shot"}. This record does not assert independent reproduction. ### KiC-Small ANLI R1 Accuracy: 33.3. Result ID: 699083733a6c314849011aaa. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Params":"0.06B","Evaluation":"Zero-shot"}. This record does not assert independent reproduction. ### T0-Base ANLI R1 Accuracy: 32.2. Result ID: 699083733a6c314849011aa6. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Params":"0.22B","Evaluation":"Zero-shot"}. This record does not assert independent reproduction. ### T0-Large ANLI R1 Accuracy: 30.5. Result ID: 699083733a6c314849011aa7. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Params":"0.77B","Evaluation":"Zero-shot"}. This record does not assert independent reproduction. ### KiC-Base ANLI R1 Accuracy: 28.4. Result ID: 699083733a6c314849011aab. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Params":"0.22B","Evaluation":"Zero-shot"}. This record does not assert independent reproduction. Page 1 of 1. This is an index page, not the full collection.