Multi-needle retrieval on NIAH (M)
90.2Accuracy (NIAH M)GLM4-9B-Chat-1M
Evaluation Results
| Method | Links | |
|---|---|---|
| GLM4-9B-Chat-1MLength=128k2024.08 | 90.2 | |
| Gradient-Llama3-8BLength=128k2024.08 | 89.6 | |
| Llama3.1-8B-InstructLength=128k2024.08 | 89 | |
| Yi-34B-200kLength=128k2024.08 | 84.9 | |
| Qwen2-72B-InstructLength=128k2024.08 | 83.4 | |
| LongRecipeModel=Llama3-8B-I, Length=128k2024.08 | 82.6 | |
| FLTModel=Llama3-8B-I, Length=80k2024.08 | 82.3 | |
| Gemini-1.5-ProLength=128k2024.08 | 82 | |
| POSEModel=Llama3-8B-I, Length=128k2024.08 | 80.1 | |
| LongRecipeModel=Qwen2-7B, Length=80k2024.08 | 79.5 | |
| Gradient-Llama3-70BLength=128k2024.08 | 79.2 | |
| LongRecipeModel=Llama3-8B-I, Length=80k2024.08 | 78.9 | |
| GPT-4Length=128k2024.08 | 76.2 | |
| RPESModel=Qwen2-7B, Length=80k2024.08 | 73.7 | |
| FLTModel=Llama3-8B-I, Length=128k2024.08 | 73.2 | |
| RPESModel=Llama3-8B-I, Length=128k2024.08 | 72.7 | |
| Llama3.1-8BLength=128k2024.08 | 72 | |
| RPESModel=Llama3-8B-I, Length=80k2024.08 | 71.8 | |
| POSEModel=Qwen2-7B, Length=80k2024.08 | 70 | |
| POSEModel=Llama3-8B-I, Length=80k2024.08 | 68.8 | |
| Llama3.1-70B-InstructLength=128k2024.08 | 68.3 | |
| LongRecipeModel=Qwen2-7B, Length=128k2024.08 | 65.8 | |
| Yi-9B-200kLength=128k2024.08 | 65.7 | |
| POSEModel=Mistral-7B, Length=80k2024.08 | 64.7 | |
| LongRecipeModel=Mistral-7B, Length=80k2024.08 | 64.7 | |
| FLT*Model=Qwen2-7B, Length=80k, Strategy=YARN and DCA2024.08 | 64.7 | |
| RPESModel=Qwen2-7B, Length=128k2024.08 | 64.6 | |
| RPESModel=Mistral-7B, Length=80k2024.08 | 60.4 | |
| POSEModel=Qwen2-7B, Length=128k2024.08 | 58.5 | |
| LongRecipeModel=Mistral-7B, Length=128k2024.08 | 53.4 | |
| FLT*Model=Qwen2-7B, Length=128k, Strategy=YARN and DCA2024.08 | 52.7 | |
| FLTModel=Mistral-7B, Length=80k2024.08 | 43 | |
| RPESModel=Mistral-7B, Length=128k2024.08 | 41.9 | |
| Qwen2-7B-InstructLength=128k2024.08 | 38.8 | |
| POSEModel=Mistral-7B, Length=128k2024.08 | 35.9 |