Multiple-Choice Question Answering on QuALITY
56.997AccuracyRAPTOR
Evaluation Results
| Method | Links | |
|---|---|---|
| RAPTORLLM=Llama-3-8B, Embedding model=BGE-M3, top-k=4, chunk size=1,200-token, Clustering method=Gaussian Mixture2025.03 | 56.997 | |
| HippoRAGLLM=Llama-3-8B, Embedding model=BGE-M3, top-k=4, chunk size=1,200-token2025.03 | 48.297 | |
| VanillaRAGLLM=Llama-3-8B, Embedding model=BGE-M3, top-k=4, chunk size=1,200-token2025.03 | 39.141 | |
| MistralTraining Protocol=Trained from scratch, Training Tokens=>1T, Input Context Length=16K2024.09 | 38 | |
| FastGraphRAGLLM=Llama-3-8B, Embedding model=BGE-M3, top-k=4, chunk size=1,200-token2025.03 | 37.275 | |
| ZeroShotLLM=Llama-3-8B2025.03 | 37.058 | |
| LGraphRAGLLM=Llama-3-8B, Embedding model=BGE-M3, top-k=4, chunk size=1,200-token2025.03 | 37.036 | |
| ToGLLM=Llama-3-8B, Embedding model=BGE-M3, top-k=4, chunk size=1,200-token2025.03 | 34.888 | |
| LLightRAGLLM=Llama-3-8B, Embedding model=BGE-M3, top-k=4, chunk size=1,200-token2025.03 | 34.78 | |
| HLightRAGLLM=Llama-3-8B, Embedding model=BGE-M3, top-k=4, chunk size=1,200-token2025.03 | 34.368 | |
| DALKLLM=Llama-3-8B, Embedding model=BGE-M3, top-k=4, chunk size=1,200-token2025.03 | 34.251 | |
| KGPLLM=Llama-3-8B, Embedding model=BGE-M3, top-k=4, chunk size=1,200-token2025.03 | 33.955 | |
| GLightRAGLLM=Llama-3-8B, Embedding model=BGE-M3, top-k=4, chunk size=1,200-token2025.03 | 33.413 | |
| GSATraining Protocol=Finetuned from Mistral 7B, Training Tokens=20B, Input Context Length=16K, Base Model=Mistral 7B2024.09 | 32 | |
| G-retrieverLLM=Llama-3-8B, Embedding model=BGE-M3, top-k=4, chunk size=1,200-token2025.03 | 31.807 | |
| GLATraining Protocol=Finetuned from Mistral 7B, Training Tokens=20B, Input Context Length=16K, Base Model=Mistral 7B2024.09 | 30.9 | |
| RWKV6Training Protocol=Trained from scratch, Training Tokens=>1T, Input Context Length=16K2024.09 | 30.8 | |
| MambaTraining Protocol=Trained from scratch, Training Tokens=>1T, Input Context Length=16K2024.09 | 27.5 | |
| RetNetTraining Protocol=Finetuned from Mistral 7B, Training Tokens=20B, Input Context Length=16K, Base Model=Mistral 7B2024.09 | 26.2 |