Passage Ranking on NQ
52.76MRREcoRank
Evaluation results
| Method | Links | ||
|---|---|---|---|
| EcoRankStrategy=Hybr., LLM=T5-XL, Budget=B1 - 0.57 cents2024.02 | 52.76 | 45.87 | |
| B-PRPStrategy=Pair, LLM=T5-XL, Budget=B1 - 0.57 cents2024.02 | 51.85 | 45.04 | |
| EcoRankStrategy=Hybr., LLM=T5-XL+L, Budget=B2 - 0.11 cents2024.02 | 50.72 | 44.34 | |
| monot5-3BLLM=T5-3B2024.02 | 50.47 | 41.66 | |
| EcoRank-w/o-casc.Strategy=Hybr., LLM=T5-XL, Budget=B2 - 0.11 cents2024.02 | 48.56 | 42.63 | |
| monot5-baseLLM=T5-base2024.02 | 48.01 | 39.11 | |
| monoBERTLLM=BERT2024.02 | 47.31 | 38.25 | |
| EcoRankStrategy=Hybr., LLM=T5-XL+L, Budget=B3 - 0.05 cents2024.02 | 46.83 | 40.33 | |
| BinaryStrategy=Point, LLM=T5-XL, Budget=B1 - 0.57 cents2024.02 | 46.77 | 37.36 | |
| TARTLLM=T5-XL2024.02 | 46.42 | 37.47 | |
| B-PRPStrategy=Pair, LLM=T5-XL, Budget=B2 - 0.11 cents2024.02 | 45.97 | 40.3 | |
| B-RankGPTStrategy=List, LLM=GPT-3.5, Budget=B1 - 0.57 cents2024.02 | 45.05 | 37.83 | |
| BinaryStrategy=Point, LLM=T5-XL, Budget=B2 - 0.11 cents2024.02 | 44.88 | 36.06 | |
| InParsLLM=T5-large, Budget=B3 - 0.05 cents2024.02 | 44.49 | 34.79 | |
| EcoRank-w/o-casc.Strategy=Hybr., LLM=T5-XL, Budget=B3 - 0.05 cents2024.02 | 44.13 | 37.89 | |
| InParsLLM=T5-large, Budget=B2 - 0.11 cents2024.02 | 43.92 | 33.57 | |
| BinaryStrategy=Point, LLM=T5-XL, Budget=B3 - 0.05 cents2024.02 | 43.06 | 34.59 | |
| B-PRPStrategy=Pair, LLM=T5-XL, Budget=B3 - 0.05 cents2024.02 | 42.81 | 36.98 | |
| InParsLLM=T5-large, Budget=B1 - 0.57 cents2024.02 | 42.73 | 32.43 | |
| LikertStrategy=Point, LLM=T5-XL, Budget=B1 - 0.57 cents2024.02 | 39.49 | 28.25 | |
| LikertStrategy=Point, LLM=T5-XL, Budget=B2 - 0.11 cents2024.02 | 38.96 | 28 | |
| LikertStrategy=Point, LLM=T5-XL, Budget=B3 - 0.05 cents2024.02 | 38.31 | 27.83 | |
| B-PRPStrategy=Pair, LLM=GPT-3.5, Budget=B1 - 0.57 cents2024.02 | 36.68 | 29 | |
| BinaryStrategy=Point, LLM=GPT-3.5, Budget=B1 - 0.57 cents2024.02 | 36.1 | 25.8 | |
| BM252024.02 | 32.49 | 22.1 | |
| OpenAI EmbeddingLLM=text-ada-002, Budget=B2 - 0.11 cents2024.02 | 32.37 | 22.1 | |
| B-UPRStrategy=Point, LLM=T5-XL, Budget=B3 - 0.05 cents2024.02 | 29.53 | 18.55 | |
| B-UPRStrategy=Point, LLM=T5-XL, Budget=B2 - 0.11 cents2024.02 | 28.03 | 17.45 | |
| B-UPRStrategy=Point, LLM=T5-XL, Budget=B1 - 0.57 cents2024.02 | 27.37 | 17.1 |