Long-context retrieval on Needle-in-the-Haystack 10k-context
100AccuracyQuest
Evaluation Results
| Method | Links | |
|---|---|---|
| QuestModel=LongChat-7B-v1.5-32k, Budget (K)=5122025.08 | 100 | |
| TidalDecodeModel=LongChat-7B-v1.5-32k, Budget (K)=5122025.08 | 100 | |
| LessIsMoreModel=LongChat-7B-v1.5-32k, Budget (K)=2562025.08 | 100 | |
| LessIsMoreModel=LongChat-7B-v1.5-32k, Budget (K)=5122025.08 | 100 | |
| QuestModel=LongChat-7B-v1.5-32k, Budget (K)=642025.08 | 99 | |
| QuestModel=LongChat-7B-v1.5-32k, Budget (K)=1282025.08 | 99 | |
| QuestModel=LongChat-7B-v1.5-32k, Budget (K)=2562025.08 | 99 | |
| TidalDecodeModel=LongChat-7B-v1.5-32k, Budget (K)=2562025.08 | 99 | |
| TidalDecodeModel=LongChat-7B-v1.5-32k, Budget (K)=1282025.08 | 98 | |
| LessIsMoreModel=LongChat-7B-v1.5-32k, Budget (K)=642025.08 | 98 | |
| LessIsMoreModel=LongChat-7B-v1.5-32k, Budget (K)=1282025.08 | 98 | |
| TidalDecodeModel=LongChat-7B-v1.5-32k, Budget (K)=642025.08 | 92 | |
| LessIsMoreModel=LongChat-7B-v1.5-32k, Budget (K)=322025.08 | 92 | |
| TidalDecodeModel=LongChat-7B-v1.5-32k, Budget (K)=322025.08 | 73 | |
| QuestModel=LongChat-7B-v1.5-32k, Budget (K)=322025.08 | 65 | |
| TOVAModel=LongChat-7B-v1.5-32k, Budget (K)=5122025.08 | 8 | |
| StreamingLLMModel=LongChat-7B-v1.5-32k, Budget (K)=5122025.08 | 5 | |
| H2OModel=LongChat-7B-v1.5-32k, Budget (K)=5122025.08 | 3 | |
| TOVAModel=LongChat-7B-v1.5-32k, Budget (K)=2562025.08 | 3 | |
| StreamingLLMModel=LongChat-7B-v1.5-32k, Budget (K)=2562025.08 | 3 | |
| H2OModel=LongChat-7B-v1.5-32k, Budget (K)=642025.08 | 1 | |
| H2OModel=LongChat-7B-v1.5-32k, Budget (K)=1282025.08 | 1 | |
| H2OModel=LongChat-7B-v1.5-32k, Budget (K)=2562025.08 | 1 | |
| TOVAModel=LongChat-7B-v1.5-32k, Budget (K)=642025.08 | 1 | |
| TOVAModel=LongChat-7B-v1.5-32k, Budget (K)=1282025.08 | 1 | |
| StreamingLLMModel=LongChat-7B-v1.5-32k, Budget (K)=322025.08 | 1 | |
| StreamingLLMModel=LongChat-7B-v1.5-32k, Budget (K)=642025.08 | 1 | |
| StreamingLLMModel=LongChat-7B-v1.5-32k, Budget (K)=1282025.08 | 1 | |
| H2OModel=LongChat-7B-v1.5-32k, Budget (K)=322025.08 | 0 | |
| TOVAModel=LongChat-7B-v1.5-32k, Budget (K)=322025.08 | 0 |