List your company
ResearchFormerly WizWand

Multiple-Choice Question Answering on QuALITY

56.997AccuracyRAPTOR
24.9681233.2833141.598549.91369Sep 11, 2024Oct 10, 2024Nov 8, 2024Dec 8, 2024Jan 6, 2025Feb 4, 2025Mar 6, 2025
Updated 6mo ago

Evaluation results

MethodLinks
RAPTORLLM=Llama-3-8B, Embedding model=BGE-M3, top-k=4, chunk size=1,200-token, Clustering method=Gaussian Mixture2025.0356.997
HippoRAGLLM=Llama-3-8B, Embedding model=BGE-M3, top-k=4, chunk size=1,200-token2025.0348.297
VanillaRAGLLM=Llama-3-8B, Embedding model=BGE-M3, top-k=4, chunk size=1,200-token2025.0339.141
MistralTraining Protocol=Trained from scratch, Training Tokens=>1T, Input Context Length=16K2024.0938
FastGraphRAGLLM=Llama-3-8B, Embedding model=BGE-M3, top-k=4, chunk size=1,200-token2025.0337.275
ZeroShotLLM=Llama-3-8B2025.0337.058
LGraphRAGLLM=Llama-3-8B, Embedding model=BGE-M3, top-k=4, chunk size=1,200-token2025.0337.036
ToGLLM=Llama-3-8B, Embedding model=BGE-M3, top-k=4, chunk size=1,200-token2025.0334.888
LLightRAGLLM=Llama-3-8B, Embedding model=BGE-M3, top-k=4, chunk size=1,200-token2025.0334.78
HLightRAGLLM=Llama-3-8B, Embedding model=BGE-M3, top-k=4, chunk size=1,200-token2025.0334.368
DALKLLM=Llama-3-8B, Embedding model=BGE-M3, top-k=4, chunk size=1,200-token2025.0334.251
KGPLLM=Llama-3-8B, Embedding model=BGE-M3, top-k=4, chunk size=1,200-token2025.0333.955
GLightRAGLLM=Llama-3-8B, Embedding model=BGE-M3, top-k=4, chunk size=1,200-token2025.0333.413
GSATraining Protocol=Finetuned from Mistral 7B, Training Tokens=20B, Input Context Length=16K, Base Model=Mistral 7B2024.0932
G-retrieverLLM=Llama-3-8B, Embedding model=BGE-M3, top-k=4, chunk size=1,200-token2025.0331.807
GLATraining Protocol=Finetuned from Mistral 7B, Training Tokens=20B, Input Context Length=16K, Base Model=Mistral 7B2024.0930.9
RWKV6Training Protocol=Trained from scratch, Training Tokens=>1T, Input Context Length=16K2024.0930.8
MambaTraining Protocol=Trained from scratch, Training Tokens=>1T, Input Context Length=16K2024.0927.5
RetNetTraining Protocol=Finetuned from Mistral 7B, Training Tokens=20B, Input Context Length=16K, Base Model=Mistral 7B2024.0926.2