Natural Language Inference on RTE Neg (accuracy)
88.1Accuracy (RTE Neg)RoBERTa-large
Evaluation results
| Method | Links | |
|---|---|---|
| RoBERTa-large# Params.=355M, Evaluation Setting=Fully supervised, Pre-training Corpus=Best of (¬ATOMIC, ¬ANION)2026.04 | 88.1 | |
| OpenAI o1# Params.=—, Evaluation Setting=In-context learning, Prompting Strategy=Zero-shot2026.04 | 87.5 | |
| RoBERTa-large-NSP# Params.=355M, Evaluation Setting=Fully supervised2026.04 | 87.2 | |
| GPT-4o# Params.=—, Evaluation Setting=In-context learning, Prompting Strategy=Few-shot2026.04 | 86.9 | |
| RoBERTa-large# Params.=355M, Evaluation Setting=Fully supervised, Pre-training Corpus=ATOMIC + ANION2026.04 | 86.2 | |
| RoBERTa-large# Params.=355M, Evaluation Setting=Fully supervised2026.04 | 84.7 | |
| Qwen2 7B# Params.=7B, Evaluation Setting=In-context learning, Prompting Strategy=Few-shot, Pre-training Corpus=Best of (¬ATOMIC, ¬ANION)2026.04 | 82.3 | |
| Llama 3.1 8B# Params.=8B, Evaluation Setting=In-context learning, Prompting Strategy=Few-shot, Pre-training Corpus=Best of (¬ATOMIC, ¬ANION)2026.04 | 81.3 | |
| Llama 3.1 70B# Params.=70B, Evaluation Setting=In-context learning, Prompting Strategy=Few-shot2026.04 | 78.9 | |
| Qwen2 7B# Params.=7B, Evaluation Setting=In-context learning, Prompting Strategy=Few-shot, Pre-training Corpus=ATOMIC + ANION2026.04 | 78.3 | |
| BERTNOT# Params.=110M, Evaluation Setting=Fully supervised2026.04 | 74.5 | |
| Qwen2 7B# Params.=7B, Evaluation Setting=In-context learning, Prompting Strategy=Few-shot2026.04 | 71.7 | |
| Llama 3.1 8B# Params.=8B, Evaluation Setting=In-context learning, Prompting Strategy=Few-shot, Pre-training Corpus=ATOMIC + ANION2026.04 | 65.3 | |
| Llama 3.1 8B# Params.=8B, Evaluation Setting=In-context learning, Prompting Strategy=Few-shot2026.04 | 60 |