List your company
ResearchFormerly WizWand

Question Answering on NQ, TriviaQA, BioASQ, and PopQA

0.617AccuracyASTUTE RAG
0.387160.446830.50650.56617Oct 9, 2024
Updated 8mo ago

Evaluation results

MethodLinks
ASTUTE RAGLLM=Claude 3.5 Sonnet (20240620), Evaluation Setting=zero-shot2024.100.617
ASTUTE RAGLLM=Mistral-Large (2407), 128B, Evaluation Setting=zero-shot2024.100.599
USCLLM=Mistral-Large (2407), 128B, Evaluation Setting=zero-shot2024.100.595
ASTUTE RAGLLM=Gemini 1.5 Pro (002), Evaluation Setting=zero-shot2024.100.592
InstructRAGLLM=Claude 3.5 Sonnet (20240620), Evaluation Setting=zero-shot2024.100.588
USCLLM=Claude 3.5 Sonnet (20240620), Evaluation Setting=zero-shot2024.100.587
Self-RouteLLM=Claude 3.5 Sonnet (20240620), Evaluation Setting=zero-shot2024.100.581
Self-RouteLLM=Gemini 1.5 Pro (002), Evaluation Setting=zero-shot2024.100.576
InstructRAGLLM=Mistral-Large (2407), 128B, Evaluation Setting=zero-shot2024.100.567
RobustRAGLLM=Claude 3.5 Sonnet (20240620), Evaluation Setting=zero-shot2024.100.565
USCLLM=Gemini 1.5 Pro (002), Evaluation Setting=zero-shot2024.100.564
Self-RouteLLM=Mistral-Large (2407), 128B, Evaluation Setting=zero-shot2024.100.562
InstructRAGLLM=Gemini 1.5 Pro (002), Evaluation Setting=zero-shot2024.100.561
RAGLLM=Claude 3.5 Sonnet (20240620), Evaluation Setting=zero-shot2024.100.555
GenReadLLM=Gemini 1.5 Pro (002), Evaluation Setting=zero-shot2024.100.547
RAGLLM=Mistral-Large (2407), 128B, Evaluation Setting=zero-shot2024.100.547
No RAGLLM=Claude 3.5 Sonnet (20240620), Evaluation Setting=zero-shot2024.100.545
RAGLLM=Gemini 1.5 Pro (002), Evaluation Setting=zero-shot2024.100.537
GenReadLLM=Claude 3.5 Sonnet (20240620), Evaluation Setting=zero-shot2024.100.536
RobustRAGLLM=Mistral-Large (2407), 128B, Evaluation Setting=zero-shot2024.100.53
GenReadLLM=Mistral-Large (2407), 128B, Evaluation Setting=zero-shot2024.100.527
No RAGLLM=Gemini 1.5 Pro (002), Evaluation Setting=zero-shot2024.100.513
ASTUTE RAGLLM=Mistral-Nemo (2407), 12B, Evaluation Setting=zero-shot2024.100.513
Self-RouteLLM=Mistral-Nemo (2407), 12B, Evaluation Setting=zero-shot2024.100.512
No RAGLLM=Mistral-Large (2407), 128B, Evaluation Setting=zero-shot2024.100.511
GenReadLLM=Mistral-Nemo (2407), 12B, Evaluation Setting=zero-shot2024.100.487
RAGLLM=Mistral-Nemo (2407), 12B, Evaluation Setting=zero-shot2024.100.483
RobustRAGLLM=Mistral-Nemo (2407), 12B, Evaluation Setting=zero-shot2024.100.464
RobustRAGLLM=Gemini 1.5 Pro (002), Evaluation Setting=zero-shot2024.100.4564
InstructRAGLLM=Mistral-Nemo (2407), 12B, Evaluation Setting=zero-shot2024.100.455
No RAGLLM=Mistral-Nemo (2407), 12B, Evaluation Setting=zero-shot2024.100.402
USCLLM=Mistral-Nemo (2407), 12B, Evaluation Setting=zero-shot2024.100.396