Question Answering on NQ, TriviaQA, BioASQ, and PopQA
0.617AccuracyASTUTE RAG
Evaluation Results
| Method | Links | |
|---|---|---|
| ASTUTE RAGLLM=Claude 3.5 Sonnet (20240620), Evaluation Setting=zero-shot2024.10 | 0.617 | |
| ASTUTE RAGLLM=Mistral-Large (2407), 128B, Evaluation Setting=zero-shot2024.10 | 0.599 | |
| USCLLM=Mistral-Large (2407), 128B, Evaluation Setting=zero-shot2024.10 | 0.595 | |
| ASTUTE RAGLLM=Gemini 1.5 Pro (002), Evaluation Setting=zero-shot2024.10 | 0.592 | |
| InstructRAGLLM=Claude 3.5 Sonnet (20240620), Evaluation Setting=zero-shot2024.10 | 0.588 | |
| USCLLM=Claude 3.5 Sonnet (20240620), Evaluation Setting=zero-shot2024.10 | 0.587 | |
| Self-RouteLLM=Claude 3.5 Sonnet (20240620), Evaluation Setting=zero-shot2024.10 | 0.581 | |
| Self-RouteLLM=Gemini 1.5 Pro (002), Evaluation Setting=zero-shot2024.10 | 0.576 | |
| InstructRAGLLM=Mistral-Large (2407), 128B, Evaluation Setting=zero-shot2024.10 | 0.567 | |
| RobustRAGLLM=Claude 3.5 Sonnet (20240620), Evaluation Setting=zero-shot2024.10 | 0.565 | |
| USCLLM=Gemini 1.5 Pro (002), Evaluation Setting=zero-shot2024.10 | 0.564 | |
| Self-RouteLLM=Mistral-Large (2407), 128B, Evaluation Setting=zero-shot2024.10 | 0.562 | |
| InstructRAGLLM=Gemini 1.5 Pro (002), Evaluation Setting=zero-shot2024.10 | 0.561 | |
| RAGLLM=Claude 3.5 Sonnet (20240620), Evaluation Setting=zero-shot2024.10 | 0.555 | |
| GenReadLLM=Gemini 1.5 Pro (002), Evaluation Setting=zero-shot2024.10 | 0.547 | |
| RAGLLM=Mistral-Large (2407), 128B, Evaluation Setting=zero-shot2024.10 | 0.547 | |
| No RAGLLM=Claude 3.5 Sonnet (20240620), Evaluation Setting=zero-shot2024.10 | 0.545 | |
| RAGLLM=Gemini 1.5 Pro (002), Evaluation Setting=zero-shot2024.10 | 0.537 | |
| GenReadLLM=Claude 3.5 Sonnet (20240620), Evaluation Setting=zero-shot2024.10 | 0.536 | |
| RobustRAGLLM=Mistral-Large (2407), 128B, Evaluation Setting=zero-shot2024.10 | 0.53 | |
| GenReadLLM=Mistral-Large (2407), 128B, Evaluation Setting=zero-shot2024.10 | 0.527 | |
| No RAGLLM=Gemini 1.5 Pro (002), Evaluation Setting=zero-shot2024.10 | 0.513 | |
| ASTUTE RAGLLM=Mistral-Nemo (2407), 12B, Evaluation Setting=zero-shot2024.10 | 0.513 | |
| Self-RouteLLM=Mistral-Nemo (2407), 12B, Evaluation Setting=zero-shot2024.10 | 0.512 | |
| No RAGLLM=Mistral-Large (2407), 128B, Evaluation Setting=zero-shot2024.10 | 0.511 | |
| GenReadLLM=Mistral-Nemo (2407), 12B, Evaluation Setting=zero-shot2024.10 | 0.487 | |
| RAGLLM=Mistral-Nemo (2407), 12B, Evaluation Setting=zero-shot2024.10 | 0.483 | |
| RobustRAGLLM=Mistral-Nemo (2407), 12B, Evaluation Setting=zero-shot2024.10 | 0.464 | |
| RobustRAGLLM=Gemini 1.5 Pro (002), Evaluation Setting=zero-shot2024.10 | 0.4564 | |
| InstructRAGLLM=Mistral-Nemo (2407), 12B, Evaluation Setting=zero-shot2024.10 | 0.455 | |
| No RAGLLM=Mistral-Nemo (2407), 12B, Evaluation Setting=zero-shot2024.10 | 0.402 | |
| USCLLM=Mistral-Nemo (2407), 12B, Evaluation Setting=zero-shot2024.10 | 0.396 |