Needle-in-the-Haystack retrieval on PG-19 mini 100K context
100AccuracyTidalDecode
Evaluation Results
| Method | Links | |
|---|---|---|
| TidalDecodeModel=Llama-3-8B, Budget (K)=1282025.08 | 100 | |
| TidalDecodeModel=Llama-3-8B, Budget (K)=2562025.08 | 100 | |
| TidalDecodeModel=Llama-3-8B, Budget (K)=5122025.08 | 100 | |
| LessIsMoreModel=Llama-3-8B, Budget (K)=642025.08 | 100 | |
| LessIsMoreModel=Llama-3-8B, Budget (K)=1282025.08 | 100 | |
| LessIsMoreModel=Llama-3-8B, Budget (K)=2562025.08 | 100 | |
| LessIsMoreModel=Llama-3-8B, Budget (K)=5122025.08 | 100 | |
| QuestModel=Llama-3-8B, Budget (K)=5122025.08 | 98 | |
| LessIsMoreModel=Llama-3-8B, Budget (K)=322025.08 | 98 | |
| TidalDecodeModel=Llama-3-8B, Budget (K)=642025.08 | 92 | |
| QuestModel=Llama-3-8B, Budget (K)=2562025.08 | 87 | |
| TidalDecodeModel=Llama-3-8B, Budget (K)=322025.08 | 86 | |
| QuestModel=Llama-3-8B, Budget (K)=1282025.08 | 65 | |
| QuestModel=Llama-3-8B, Budget (K)=642025.08 | 50 | |
| QuestModel=Llama-3-8B, Budget (K)=322025.08 | 38 |