Re-ranking on LEO dataset
60.7nDCG@1ASTRA
Evaluation results
| Method | Links | |||||||||
|---|---|---|---|---|---|---|---|---|---|---|
| ASTRACategory=Spatiotemporal-Aware, Latency=1.80, Cost=10.02026.06 | 60.7 | 57.2 | 63.8 | 60.7 | 71.1 | 72.1 | 24.7 | 46.7 | 51.8 | |
| stage 1 retrievalCategory=N/A2026.06 | 43.2 | 45.3 | 51.1 | 43.2 | 56 | 57.2 | 18.2 | 36.1 | 40.1 | |
| LLMRerank (gpt-3.5)Category=LLM as Re-Ranker, Latency=14.47, Cost=15.02026.06 | 41.3 | 35.5 | 40.3 | 41.3 | 49.9 | 51.5 | 41.3 | 48.1 | 46.6 | |
| RankGPT (gpt-4o-mini)Category=LLM as Re-Ranker, Latency=23.11, Cost=0.42026.06 | 36.3 | 42.6 | 48 | 36.3 | 51.1 | 52.4 | 36.3 | 49.9 | 48.8 | |
| SBERT-MiniLM (L-6-v2)Category=Cross-Encoder, Latency=0.12, Cost=free2026.06 | 34.5 | 35.6 | 41 | 34.5 | 45.4 | 46.9 | 34.5 | 43.8 | 43.1 | |
| ColBERTCategory=Late Interaction, Latency=7.26, Cost=free2026.06 | 33.6 | 31.7 | 36.7 | 33.6 | 43.3 | 45.2 | 33.6 | 41.7 | 41 | |
| RRFCategory=Spatiotemporal-Aware, Latency=1.80, Cost=10.02026.06 | 33.1 | 36.1 | 42.9 | 33.1 | 47 | 48.7 | 13 | 26.9 | 31 | |
| LLMRerank (gpt-4o-mini)Category=LLM as Re-Ranker, Latency=17.39, Cost=0.92026.06 | 32.9 | 29 | 33 | 32.9 | 42 | 43.6 | 32.9 | 40.8 | 39.6 | |
| GT-RCategory=Spatiotemporal-Aware, Latency=1.56, Cost=3.32026.06 | 31.1 | 33.3 | 38.9 | 31.1 | 33.3 | 38.9 | 31.1 | 33.3 | 40.4 | |
| Jina_reranker (v2)Category=Cross-Encoder, Latency=0.59, Cost=0.162026.06 | 30.9 | 32.1 | 37.6 | 30.9 | 39.8 | 41.1 | 30.9 | 38.2 | 36.8 | |
| Cohere_rerank (v3)Category=Cross-Encoder, Latency=0.23, Cost=2.02026.06 | 29.5 | 28.7 | 33.6 | 29.5 | 38.9 | 40.7 | 29.5 | 37.6 | 37.2 | |
| BGE-reranker (v2-m3)Category=Cross-Encoder, Latency=3.60, Cost=free2026.06 | 26.4 | 28.5 | 34.1 | 26.4 | 37.9 | 39.8 | 26.4 | 36.3 | 35.7 | |
| BGE-reranker (base)Category=Cross-Encoder, Latency=1.20, Cost=free2026.06 | 17.5 | 20.2 | 24.9 | 17.5 | 27.2 | 29.2 | 17.5 | 26.1 | 26.8 |