Needle-in-the-Haystack retrieval on PG-19 mini 10K context
100Accuracy (Needle-in-the-Haystack)Quest
Evaluation Results
| Method | Links | |
|---|---|---|
| QuestModel=Llama-3-8B, Budget (K)=5122025.08 | 100 | |
| TidalDecodeModel=Llama-3-8B, Budget (K)=1282025.08 | 100 | |
| TidalDecodeModel=Llama-3-8B, Budget (K)=2562025.08 | 100 | |
| TidalDecodeModel=Llama-3-8B, Budget (K)=5122025.08 | 100 | |
| LessIsMoreModel=Llama-3-8B, Budget (K)=322025.08 | 100 | |
| LessIsMoreModel=Llama-3-8B, Budget (K)=642025.08 | 100 | |
| LessIsMoreModel=Llama-3-8B, Budget (K)=1282025.08 | 100 | |
| LessIsMoreModel=Llama-3-8B, Budget (K)=2562025.08 | 100 | |
| LessIsMoreModel=Llama-3-8B, Budget (K)=5122025.08 | 100 | |
| QuestModel=Llama-3.1-8B, Budget (K)=2562025.08 | 100 | |
| TidalDecodeModel=Llama-3.1-8B, Budget (K)=322025.08 | 100 | |
| TidalDecodeModel=Llama-3.1-8B, Budget (K)=642025.08 | 100 | |
| TidalDecodeModel=Llama-3.1-8B, Budget (K)=1282025.08 | 100 | |
| TidalDecodeModel=Llama-3.1-8B, Budget (K)=2562025.08 | 100 | |
| TidalDecodeModel=Llama-3.1-8B, Budget (K)=5122025.08 | 100 | |
| LessIsMoreModel=Llama-3.1-8B, Budget (K)=322025.08 | 100 | |
| LessIsMoreModel=Llama-3.1-8B, Budget (K)=642025.08 | 100 | |
| LessIsMoreModel=Llama-3.1-8B, Budget (K)=1282025.08 | 100 | |
| LessIsMoreModel=Llama-3.1-8B, Budget (K)=2562025.08 | 100 | |
| LessIsMoreModel=Llama-3.1-8B, Budget (K)=5122025.08 | 100 | |
| QuestModel=Llama-3-8B, Budget (K)=1282025.08 | 99 | |
| QuestModel=Llama-3-8B, Budget (K)=2562025.08 | 98 | |
| TidalDecodeModel=Llama-3-8B, Budget (K)=642025.08 | 98 | |
| QuestModel=Llama-3.1-8B, Budget (K)=5122025.08 | 98 | |
| QuestModel=Llama-3.1-8B, Budget (K)=1282025.08 | 94 | |
| TidalDecodeModel=Llama-3-8B, Budget (K)=322025.08 | 88 | |
| QuestModel=Llama-3.1-8B, Budget (K)=642025.08 | 86 | |
| QuestModel=Llama-3-8B, Budget (K)=642025.08 | 84 | |
| QuestModel=Llama-3-8B, Budget (K)=322025.08 | 74 | |
| QuestModel=Llama-3.1-8B, Budget (K)=322025.08 | 74 |