Structured Information Retrieval on LCR, WikiText, and SWDE Average
87F1 ScoreAnnoRetrieve
Evaluation results
| Method | Links | |
|---|---|---|
| AnnoRetrieveLLM Cost (K Tokens)=29.4, Latency (s)=3.22026.04 | 87 | |
| QUESTLLM Cost (K Tokens)=22.1, Latency (s)=4.02026.04 | 78 | |
| LLM(GPT-4)LLM Cost (K Tokens)=312.8, Latency (s)=24.72026.04 | 72 | |
| PalimpsestLLM Cost (K Tokens)=38.9, Latency (s)=5.32026.04 | 70 | |
| ZenDBLLM Cost (K Tokens)=45.6, Latency (s)=6.12026.04 | 65 | |
| Graph RAGLLM Cost (K Tokens)=142.5 (Graph Build) + 8.7, Latency (s)=12.42026.04 | 58 | |
| ClosedIELatency (s)=0.52026.04 | 49 | |
| VectorDBLLM Cost (K Tokens)=15.2, Latency (s)=1.82026.04 | 41 |