Natural Language Inference on SNLI-Neg
75.9AccuracyOpenAI o1
Evaluation Results
| Method | Links | |
|---|---|---|
| OpenAI o1# Params.=—, Evaluation Setting=In-context learning, Prompting Strategy=Zero-shot2026.04 | 75.9 | |
| GPT-4o# Params.=—, Evaluation Setting=In-context learning, Prompting Strategy=Few-shot2026.04 | 74.8 | |
| Qwen2 7B# Params.=7B, Evaluation Setting=In-context learning, Prompting Strategy=Few-shot, Pre-training Corpus=Best of (¬ATOMIC, ¬ANION)2026.04 | 72.4 | |
| Llama 3.1 70B# Params.=70B, Evaluation Setting=In-context learning, Prompting Strategy=Few-shot2026.04 | 69.1 | |
| Llama 3.1 8B# Params.=8B, Evaluation Setting=In-context learning, Prompting Strategy=Few-shot, Pre-training Corpus=Best of (¬ATOMIC, ¬ANION)2026.04 | 68.2 | |
| Qwen2 7B# Params.=7B, Evaluation Setting=In-context learning, Prompting Strategy=Few-shot, Pre-training Corpus=ATOMIC + ANION2026.04 | 66.5 | |
| Qwen2 7B# Params.=7B, Evaluation Setting=In-context learning, Prompting Strategy=Few-shot2026.04 | 60.7 | |
| RoBERTa-large# Params.=355M, Evaluation Setting=Fully supervised, Pre-training Corpus=Best of (¬ATOMIC, ¬ANION)2026.04 | 58.3 | |
| Llama 3.1 8B# Params.=8B, Evaluation Setting=In-context learning, Prompting Strategy=Few-shot, Pre-training Corpus=ATOMIC + ANION2026.04 | 57.5 | |
| RoBERTa-large-NSP# Params.=355M, Evaluation Setting=Fully supervised2026.04 | 56.5 | |
| RoBERTa-large# Params.=355M, Evaluation Setting=Fully supervised, Pre-training Corpus=ATOMIC + ANION2026.04 | 56.5 | |
| RoBERTa-large# Params.=355M, Evaluation Setting=Fully supervised2026.04 | 56 | |
| Llama 3.1 8B# Params.=8B, Evaluation Setting=In-context learning, Prompting Strategy=Few-shot2026.04 | 54.4 | |
| BERTNOT# Params.=110M, Evaluation Setting=Fully supervised2026.04 | 46 |