List your company
ResearchFormerly WizWand

Multi-needle retrieval on NIAH (M)

90.2Accuracy (NIAH M)GLM4-9B-Chat-1M
33.72848.38963.0577.711Aug 31, 2024
Updated 8mo ago

Evaluation results

MethodLinks
GLM4-9B-Chat-1MLength=128k2024.0890.2
Gradient-Llama3-8BLength=128k2024.0889.6
Llama3.1-8B-InstructLength=128k2024.0889
Yi-34B-200kLength=128k2024.0884.9
Qwen2-72B-InstructLength=128k2024.0883.4
LongRecipeModel=Llama3-8B-I, Length=128k2024.0882.6
FLTModel=Llama3-8B-I, Length=80k2024.0882.3
Gemini-1.5-ProLength=128k2024.0882
POSEModel=Llama3-8B-I, Length=128k2024.0880.1
LongRecipeModel=Qwen2-7B, Length=80k2024.0879.5
Gradient-Llama3-70BLength=128k2024.0879.2
LongRecipeModel=Llama3-8B-I, Length=80k2024.0878.9
GPT-4Length=128k2024.0876.2
RPESModel=Qwen2-7B, Length=80k2024.0873.7
FLTModel=Llama3-8B-I, Length=128k2024.0873.2
RPESModel=Llama3-8B-I, Length=128k2024.0872.7
Llama3.1-8BLength=128k2024.0872
RPESModel=Llama3-8B-I, Length=80k2024.0871.8
POSEModel=Qwen2-7B, Length=80k2024.0870
POSEModel=Llama3-8B-I, Length=80k2024.0868.8
Llama3.1-70B-InstructLength=128k2024.0868.3
LongRecipeModel=Qwen2-7B, Length=128k2024.0865.8
Yi-9B-200kLength=128k2024.0865.7
POSEModel=Mistral-7B, Length=80k2024.0864.7
LongRecipeModel=Mistral-7B, Length=80k2024.0864.7
FLT*Model=Qwen2-7B, Length=80k, Strategy=YARN and DCA2024.0864.7
RPESModel=Qwen2-7B, Length=128k2024.0864.6
RPESModel=Mistral-7B, Length=80k2024.0860.4
POSEModel=Qwen2-7B, Length=128k2024.0858.5
LongRecipeModel=Mistral-7B, Length=128k2024.0853.4
FLT*Model=Qwen2-7B, Length=128k, Strategy=YARN and DCA2024.0852.7
FLTModel=Mistral-7B, Length=80k2024.0843
RPESModel=Mistral-7B, Length=128k2024.0841.9
Qwen2-7B-InstructLength=128k2024.0838.8
POSEModel=Mistral-7B, Length=128k2024.0835.9