# Natural Language Inference on XNLI French (test)

Source: [Natural Language Inference on XNLI French (test)](<https://www.sota2.com/research/sota/natural-language-inference-on-xnli-french-test>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2026-02-26T05:27:41.916Z

Evaluation of Natural Language Inference performance specifically on the French subset of the XNLI test set, typically measured by Accuracy.

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

| Scope | Recorded value |
| --- | --- |
| Task | Natural Language Inference |
| Dataset (including any stated split) | XNLI French (test) |
| Metric | Accuracy |
| Unit | PERCENT |
| Direction | Higher is better |
| Results loaded | 16 of 16 |
| Coverage | All results currently returned for this benchmark |

Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol.

## Metrics

- [Accuracy](<https://www.sota2.com/research/sota/natural-language-inference-on-xnli-french-test.md?metric=Accuracy>): Higher is better; unit: PERCENT.

## Recorded results

| Method | Accuracy | Result date | Report paper |
| --- | --- | --- | --- |
| CamemBERT\_LARGE | 85.7 | Not recorded | 68fa68341e209131cea1abd9 |
| XLM-R | 85.2 | Not recorded | 68fa67391e209131cea15043 |
| XLM-R\_LARGE | 85.2 | Not recorded | 68fa68341e209131cea1abd9 |
| FlauBERT | 83.4 | Not recorded | 68fa67391e209131cea15043 |
| CamemBERT | 82.5 | Not recorded | 68fa68341e209131cea1abd9 |
| CamemBERT | 81.2 | Not recorded | 68fa67391e209131cea15043 |
| FlauBERT | 80.6 | Not recorded | 68fa67391e209131cea15043 |
| XLM\_MLM-TLM | 80.2 | Not recorded | 68fa68341e209131cea1abd9 |
| XLM-R | 80.1 | Not recorded | 68fa67391e209131cea15043 |
| XLM-R\_BASE | 80.1 | Not recorded | 68fa68341e209131cea1abd9 |
| mBERT | 76.9 | Not recorded | 68fa67391e209131cea15043 |
| mBERT | 76.9 | Not recorded | 68fa68341e209131cea1abd9 |
| GenKnowSub | 37.5 | Not recorded | 68fa7a661e209131cea82edb |
| Phi-2 | 34.16 | Not recorded | 68fa7a661e209131cea82edb |
| Arrow | 34.16 | Not recorded | 68fa7a661e209131cea82edb |
| GenKnowSub | 31.83 | Not recorded | 68fa7a661e209131cea82edb |

### CamemBERT\_LARGE

Accuracy: 85.7. Result ID: 698d626b8b517eb43321184e.

[Reporting paper](<https://www.sota2.com/research/paper/68fa68341e209131cea1abd9.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa68341e209131cea1abd9.md>)

[Implementation](<https://camembert-model.fr/>)

Recorded evaluation configuration: {"mode":"fine-tuned","number of parameters":"335M"}. This record does not assert independent reproduction.

### XLM-R

Accuracy: 85.2. Result ID: 698d22be8b517eb4331f45fd.

[Reporting paper](<https://www.sota2.com/research/paper/68fa67391e209131cea15043.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa67521e209131cea15706.md>)

Recorded evaluation configuration: {"Model Size":"LARGE"}. This record does not assert independent reproduction.

### XLM-R\_LARGE

Accuracy: 85.2. Result ID: 698d626b8b517eb43321184d.

[Reporting paper](<https://www.sota2.com/research/paper/68fa68341e209131cea1abd9.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa67521e209131cea15706.md>)

Recorded evaluation configuration: {"number of parameters":"550M"}. This record does not assert independent reproduction.

### FlauBERT

Accuracy: 83.4. Result ID: 698d22be8b517eb4331f4602.

[Reporting paper](<https://www.sota2.com/research/paper/68fa67391e209131cea15043.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa67391e209131cea15043.md>)

[Implementation](<https://github.com/getalp/Flaubert>)

Recorded evaluation configuration: {"Model Size":"LARGE"}. This record does not assert independent reproduction.

### CamemBERT

Accuracy: 82.5. Result ID: 698d626b8b517eb43321184c.

[Reporting paper](<https://www.sota2.com/research/paper/68fa68341e209131cea1abd9.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa68341e209131cea1abd9.md>)

[Implementation](<https://camembert-model.fr/>)

Recorded evaluation configuration: {"mode":"fine-tuned","number of parameters":"110M"}. This record does not assert independent reproduction.

### CamemBERT

Accuracy: 81.2. Result ID: 698d22be8b517eb4331f4600.

[Reporting paper](<https://www.sota2.com/research/paper/68fa67391e209131cea15043.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa68341e209131cea1abd9.md>)

[Implementation](<https://camembert-model.fr/>)

Recorded evaluation configuration: Not recorded. This record does not assert independent reproduction.

### FlauBERT

Accuracy: 80.6. Result ID: 698d22be8b517eb4331f4601.

[Reporting paper](<https://www.sota2.com/research/paper/68fa67391e209131cea15043.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa67391e209131cea15043.md>)

[Implementation](<https://github.com/getalp/Flaubert>)

Recorded evaluation configuration: {"Model Size":"BASE"}. This record does not assert independent reproduction.

### XLM\_MLM-TLM

Accuracy: 80.2. Result ID: 698d626b8b517eb43321184a.

[Reporting paper](<https://www.sota2.com/research/paper/68fa68341e209131cea1abd9.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa65491e209131cea0943c.md>)

Recorded evaluation configuration: {"number of parameters":"250M"}. This record does not assert independent reproduction.

### XLM-R

Accuracy: 80.1. Result ID: 698d22be8b517eb4331f45fe.

[Reporting paper](<https://www.sota2.com/research/paper/68fa67391e209131cea15043.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa67521e209131cea15706.md>)

Recorded evaluation configuration: {"Model Size":"BASE"}. This record does not assert independent reproduction.

### XLM-R\_BASE

Accuracy: 80.1. Result ID: 698d626b8b517eb43321184b.

[Reporting paper](<https://www.sota2.com/research/paper/68fa68341e209131cea1abd9.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa67521e209131cea15706.md>)

Recorded evaluation configuration: {"number of parameters":"270M"}. This record does not assert independent reproduction.

### mBERT

Accuracy: 76.9. Result ID: 698d22be8b517eb4331f45ff.

[Reporting paper](<https://www.sota2.com/research/paper/68fa67391e209131cea15043.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa65c31e209131cea0c4ed.md>)

[Implementation](<https://github.com/google-research/bert>)

Recorded evaluation configuration: Not recorded. This record does not assert independent reproduction.

### mBERT

Accuracy: 76.9. Result ID: 698d626b8b517eb433211849.

[Reporting paper](<https://www.sota2.com/research/paper/68fa68341e209131cea1abd9.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa65c31e209131cea0c4ed.md>)

[Implementation](<https://github.com/google-research/bert>)

Recorded evaluation configuration: {"number of parameters":"175M"}. This record does not assert independent reproduction.

### GenKnowSub

Accuracy: 37.5. Result ID: 698b76e27a967eddb76ecd3b.

[Reporting paper](<https://www.sota2.com/research/paper/68fa7a661e209131cea82edb.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa7a661e209131cea82edb.md>)

[Implementation](<https://github.com/saharsamr/Modular-LLM>)

Recorded evaluation configuration: {"Base Model":"Phi-2","Evaluation Setting":"Zero-shot","Setting":"Avg"}. This record does not assert independent reproduction.

### Phi-2

Accuracy: 34.16. Result ID: 698b76e27a967eddb76ecd38.

[Reporting paper](<https://www.sota2.com/research/paper/68fa7a661e209131cea82edb.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa73471e209131cea5b839.md>)

Recorded evaluation configuration: {"Base Model":"Phi-2","Evaluation Setting":"Zero-shot"}. This record does not assert independent reproduction.

### Arrow

Accuracy: 34.16. Result ID: 698b76e27a967eddb76ecd39.

[Reporting paper](<https://www.sota2.com/research/paper/68fa7a661e209131cea82edb.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa73471e209131cea5b839.md>)

Recorded evaluation configuration: {"Base Model":"Phi-2","Evaluation Setting":"Zero-shot"}. This record does not assert independent reproduction.

### GenKnowSub

Accuracy: 31.83. Result ID: 698b76e27a967eddb76ecd3a.

[Reporting paper](<https://www.sota2.com/research/paper/68fa7a661e209131cea82edb.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa7a661e209131cea82edb.md>)

[Implementation](<https://github.com/saharsamr/Modular-LLM>)

Recorded evaluation configuration: {"Base Model":"Phi-2","Evaluation Setting":"Zero-shot","Setting":"En"}. This record does not assert independent reproduction.

Page 1 of 1. This is an index page, not the full collection.
