List your company
ResearchFormerly WizWand

Information Retrieval Reasoning on NevIR

0.617Pairwise AccuracyGPT-4o
0.230120.330560.4310.53144Apr 21, 2026
Updated 6mo ago

Evaluation results

MethodLinks
GPT-4o# Params.=—, Evaluation Setting=In-context learning, Prompting Strategy=Few-shot2026.040.617
OpenAI o1# Params.=—, Evaluation Setting=In-context learning, Prompting Strategy=Zero-shot2026.040.597
Llama 3.1 70B# Params.=70B, Evaluation Setting=In-context learning, Prompting Strategy=Few-shot2026.040.588
MonoT5 3B# Params.=3B, Evaluation Setting=Fully supervised2026.040.506
Llama 3.1 8B# Params.=8B, Evaluation Setting=In-context learning, Prompting Strategy=Few-shot, Pre-training Corpus=Best of (¬ATOMIC, ¬ANION)2026.040.422
Qwen2 7B# Params.=7B, Evaluation Setting=In-context learning, Prompting Strategy=Few-shot, Pre-training Corpus=Best of (¬ATOMIC, ¬ANION)2026.040.398
Llama 3.1 8B# Params.=8B, Evaluation Setting=In-context learning, Prompting Strategy=Few-shot, Pre-training Corpus=ATOMIC + ANION2026.040.379
RoBERTa-large# Params.=355M, Evaluation Setting=Fully supervised, Pre-training Corpus=Best of (¬ATOMIC, ¬ANION)2026.040.343
Qwen2 7B# Params.=7B, Evaluation Setting=In-context learning, Prompting Strategy=Few-shot, Pre-training Corpus=ATOMIC + ANION2026.040.338
Llama 3.1 8B# Params.=8B, Evaluation Setting=In-context learning, Prompting Strategy=Few-shot2026.040.306
Qwen2 7B# Params.=7B, Evaluation Setting=In-context learning, Prompting Strategy=Few-shot2026.040.298
RoBERTa-large# Params.=355M, Evaluation Setting=Fully supervised, Pre-training Corpus=ATOMIC + ANION2026.040.291
stsb-roberta-large# Params.=355M, Evaluation Setting=Fully supervised2026.040.249
RoBERTa-large# Params.=355M, Evaluation Setting=Fully supervised2026.040.245