Passage Ranking on WebQuestions (WQ)
63.97R@10monot5-3B
Evaluation Results
| Method | Links | |||
|---|---|---|---|---|
| monot5-3BSupervision=Supervised, LLM=T5-3B2024.02 | 63.97 | — | — | |
| EcoRankSupervision=Unsupervised, Budget=B1, Strategy=Hybr, LLM=T5-XL2024.02 | 63.58 | — | — | |
| monot5-baseSupervision=Supervised, LLM=T5-base2024.02 | 62.94 | — | — | |
| monoBERTSupervision=Supervised, LLM=BERT2024.02 | 62.94 | — | — | |
| TARTSupervision=Supervised, LLM=T5-XL2024.02 | 62.64 | — | — | |
| InParsSupervision=Unsupervised, Budget=B3, LLM=T5-large2024.02 | 62.25 | — | — | |
| LikertSupervision=Unsupervised, Budget=B1, Strategy=Point, LLM=T5-XL2024.02 | 62.05 | — | — | |
| InParsSupervision=Unsupervised, Budget=B1, LLM=T5-large2024.02 | 61.95 | — | — | |
| BinarySupervision=Unsupervised, Budget=B1, Strategy=Point, LLM=T5-XL2024.02 | 61.76 | — | — | |
| InParsSupervision=Unsupervised, Budget=B2, LLM=T5-large2024.02 | 61.66 | — | — | |
| LikertSupervision=Unsupervised, Budget=B2, Strategy=Point, LLM=T5-XL2024.02 | 60.92 | — | — | |
| B-PRPSupervision=Unsupervised, Budget=B1, Strategy=Pair, LLM=T5-XL2024.02 | 60.53 | — | — | |
| EcoRankSupervision=Unsupervised, Budget=B2, Strategy=Hybr, LLM=T5-XL+L2024.02 | 60.33 | — | — | |
| BinarySupervision=Unsupervised, Budget=B2, Strategy=Point, LLM=T5-XL2024.02 | 59.64 | — | — | |
| LikertSupervision=Unsupervised, Budget=B3, Strategy=Point, LLM=T5-XL2024.02 | 57.77 | — | — | |
| BinarySupervision=Unsupervised, Budget=B3, Strategy=Point, LLM=T5-XL2024.02 | 57.08 | — | — | |
| B-RankGPTSupervision=Unsupervised, Budget=B1, Strategy=List, LLM=GPT-3.52024.02 | 56.88 | — | — | |
| EcoRank-w/o-casc.Supervision=Unsupervised, Budget=B2, Strategy=Hybr, LLM=T5-XL2024.02 | 56.79 | — | — | |
| BinarySupervision=Unsupervised, Budget=B1, Strategy=Point, LLM=GPT-3.52024.02 | 56.74 | — | — | |
| EcoRankSupervision=Unsupervised, Budget=B3, Strategy=Hybr, LLM=T5-XL+L2024.02 | 56.69 | — | — | |
| B-PRPSupervision=Unsupervised, Budget=B2, Strategy=Pair, LLM=T5-XL2024.02 | 55.65 | — | — | |
| EcoRank-w/o-casc.Supervision=Unsupervised, Budget=B3, Strategy=Hybr, LLM=T5-XL2024.02 | 55.21 | — | — | |
| B-UPRSupervision=Unsupervised, Budget=B3, Strategy=Point, LLM=T5-XL2024.02 | 54.8 | — | — | |
| B-PRPSupervision=Unsupervised, Budget=B3, Strategy=Pair, LLM=T5-XL2024.02 | 53.65 | — | — | |
| BM25Supervision=None2024.02 | 52.16 | — | — | |
| B-UPRSupervision=Unsupervised, Budget=B1, Strategy=Point, LLM=T5-XL2024.02 | 52.16 | — | — | |
| B-PRPSupervision=Unsupervised, Budget=B1, Strategy=Pair, LLM=GPT-3.52024.02 | 52.16 | — | — | |
| B-UPRSupervision=Unsupervised, Budget=B2, Strategy=Point, LLM=T5-XL2024.02 | 52.16 | — | — | |
| BM252024.02 | — | 29.69 | 18.89 | |
| EcoRankStrategy=Hybr., LLM=T5-XL, Budget=B1 - 0.57 cents2024.02 | — | 48.94 | 41.58 | |
| EcoRankStrategy=Hybr., LLM=T5-XL+L, Budget=B2 - 0.11 cents2024.02 | — | 47.05 | 40.55 | |
| EcoRankStrategy=Hybr., LLM=T5-XL+L, Budget=B3 - 0.05 cents2024.02 | — | 43.86 | 37 | |
| monoBERTLLM=BERT2024.02 | — | 43.63 | 33.8 | |
| monot5-3BLLM=T5-3B2024.02 | — | 44.88 | 35.23 | |
| monot5-baseLLM=T5-base2024.02 | — | 43.15 | 33.21 | |
| TARTLLM=T5-XL2024.02 | — | 42.65 | 33.07 |