Needle-in-the-Haystack retrieval on PG-19 mini 32K context
100AccuracyQuest
Evaluation Results
| Method | Links | |
|---|---|---|
| QuestModel=Llama-3.1-8B, Budget (K)=2562025.08 | 100 | |
| QuestModel=Llama-3.1-8B, Budget (K)=5122025.08 | 100 | |
| TidalDecodeModel=Llama-3.1-8B, Budget (K)=642025.08 | 100 | |
| TidalDecodeModel=Llama-3.1-8B, Budget (K)=1282025.08 | 100 | |
| TidalDecodeModel=Llama-3.1-8B, Budget (K)=2562025.08 | 100 | |
| TidalDecodeModel=Llama-3.1-8B, Budget (K)=5122025.08 | 100 | |
| LessIsMoreModel=Llama-3.1-8B, Budget (K)=322025.08 | 100 | |
| LessIsMoreModel=Llama-3.1-8B, Budget (K)=642025.08 | 100 | |
| LessIsMoreModel=Llama-3.1-8B, Budget (K)=1282025.08 | 100 | |
| LessIsMoreModel=Llama-3.1-8B, Budget (K)=2562025.08 | 100 | |
| LessIsMoreModel=Llama-3.1-8B, Budget (K)=5122025.08 | 100 | |
| TidalDecodeModel=Llama-3.1-8B, Budget (K)=322025.08 | 98 | |
| QuestModel=Llama-3.1-8B, Budget (K)=1282025.08 | 92 | |
| QuestModel=Llama-3.1-8B, Budget (K)=642025.08 | 88 | |
| QuestModel=Llama-3.1-8B, Budget (K)=322025.08 | 78 |