Information Retrieval Reasoning on NevIR
0.617Pairwise AccuracyGPT-4o
Evaluation Results
| Method | Links | |
|---|---|---|
| GPT-4o# Params.=—, Evaluation Setting=In-context learning, Prompting Strategy=Few-shot2026.04 | 0.617 | |
| OpenAI o1# Params.=—, Evaluation Setting=In-context learning, Prompting Strategy=Zero-shot2026.04 | 0.597 | |
| Llama 3.1 70B# Params.=70B, Evaluation Setting=In-context learning, Prompting Strategy=Few-shot2026.04 | 0.588 | |
| MonoT5 3B# Params.=3B, Evaluation Setting=Fully supervised2026.04 | 0.506 | |
| Llama 3.1 8B# Params.=8B, Evaluation Setting=In-context learning, Prompting Strategy=Few-shot, Pre-training Corpus=Best of (¬ATOMIC, ¬ANION)2026.04 | 0.422 | |
| Qwen2 7B# Params.=7B, Evaluation Setting=In-context learning, Prompting Strategy=Few-shot, Pre-training Corpus=Best of (¬ATOMIC, ¬ANION)2026.04 | 0.398 | |
| Llama 3.1 8B# Params.=8B, Evaluation Setting=In-context learning, Prompting Strategy=Few-shot, Pre-training Corpus=ATOMIC + ANION2026.04 | 0.379 | |
| RoBERTa-large# Params.=355M, Evaluation Setting=Fully supervised, Pre-training Corpus=Best of (¬ATOMIC, ¬ANION)2026.04 | 0.343 | |
| Qwen2 7B# Params.=7B, Evaluation Setting=In-context learning, Prompting Strategy=Few-shot, Pre-training Corpus=ATOMIC + ANION2026.04 | 0.338 | |
| Llama 3.1 8B# Params.=8B, Evaluation Setting=In-context learning, Prompting Strategy=Few-shot2026.04 | 0.306 | |
| Qwen2 7B# Params.=7B, Evaluation Setting=In-context learning, Prompting Strategy=Few-shot2026.04 | 0.298 | |
| RoBERTa-large# Params.=355M, Evaluation Setting=Fully supervised, Pre-training Corpus=ATOMIC + ANION2026.04 | 0.291 | |
| stsb-roberta-large# Params.=355M, Evaluation Setting=Fully supervised2026.04 | 0.249 | |
| RoBERTa-large# Params.=355M, Evaluation Setting=Fully supervised2026.04 | 0.245 |