Reranking on BEIR (test)
78.5Covid ScoreMohajer+Bubble (Randomized)
Evaluation results
| Method | Links | ||||||||
|---|---|---|---|---|---|---|---|---|---|
| Mohajer+Bubble (Randomized)Backbone=Flan-T5-XL, Avg. Calls/Task=3452026.05 | 78.5 | 54 | 27.9 | 63.5 | 41.2 | 69.5 | 66.3 | 57.3 | |
| HeapSort (Bidirectional)Backbone=Flan-T5-XL, Avg. Calls/Task=14092026.05 | 78.2 | 54.9 | 28.4 | 70.6 | 41.6 | 70.6 | 68.9 | 59 | |
| Mohajer (Randomized)Backbone=Flan-T5-XL, Avg. Calls/Task=2322026.05 | 77.6 | 53.2 | 27.2 | 62.8 | 40.4 | 68.7 | 67.6 | 56.8 | |
| QuickSort (Bidirectional)Backbone=Flan-T5-XL, Avg. Calls/Task=16692026.05 | 77.2 | 53.7 | 25.8 | 61.4 | 41.6 | 70.4 | 67.2 | 56.8 | |
| Mohajer+Bubble (Randomized)Backbone=Flan-T5-L, Avg. Calls/Task=3542026.05 | 76.9 | 37.8 | 25.7 | 58.8 | 40 | 61.8 | 58.8 | 51.4 | |
| Mohajer+Bubble (Bidirectional)Backbone=Flan-T5-L, Avg. Calls/Task=4232026.05 | 76.5 | 37.8 | 26.7 | 54.9 | 40 | 63 | 56.4 | 50.7 | |
| Mohajer (Bidirectional)Backbone=Flan-T5-L, Avg. Calls/Task=3992026.05 | 76.5 | 37.5 | 26.4 | 53.8 | 39.7 | 62.6 | 56.1 | 50.4 | |
| QuickSort (Bidirectional)Backbone=Flan-T5-L, Avg. Calls/Task=19542026.05 | 76.2 | 41 | 27.4 | 60.1 | 41.1 | 64.5 | 58.5 | 52.7 | |
| Mohajer (Randomized)Backbone=Flan-T5-L, Avg. Calls/Task=2322026.05 | 76.2 | 36.2 | 24.4 | 57.5 | 39.1 | 60.6 | 57.2 | 50.2 | |
| HeapSort (Bidirectional)Backbone=Flan-T5-L, Avg. Calls/Task=12302026.05 | 76 | 40.4 | 33.2 | 67.5 | 41.4 | 65 | 62.6 | 55.2 | |
| Mohajer+Bubble (Bidirectional)Backbone=Flan-T5-XL, Avg. Calls/Task=4272026.05 | 76 | 53.7 | 25.7 | 61.9 | 40.9 | 66.6 | 67.5 | 56 | |
| Mohajer (Bidirectional)Backbone=Flan-T5-XL, Avg. Calls/Task=3992026.05 | 76 | 53.6 | 25.4 | 61.2 | 40.7 | 66.6 | 67.3 | 55.8 | |
| BubbleSort@10 (Bidirectional)Backbone=Flan-T5-XL, Avg. Calls/Task=9412026.05 | 74.8 | 55.4 | 42.8 | 71.3 | 43.1 | 68.4 | 67 | 60.4 | |
| PAC+Bubble (Bidirectional)Backbone=Flan-T5-XL, Avg. Calls/Task=3322026.05 | 71.3 | 48.9 | 41.1 | 70.4 | 39.1 | 62.6 | 58.6 | 56 | |
| PAC+Bubble (Randomized)Backbone=Flan-T5-XL, Avg. Calls/Task=1842026.05 | 71.3 | 48.1 | 38.8 | 68.6 | 38.8 | 61.1 | 58.5 | 55 | |
| BubbleSort@10 (Bidirectional)Backbone=Flan-T5-L, Avg. Calls/Task=6792026.05 | 70.9 | 44.2 | 44.7 | 69.2 | 41.7 | 63.4 | 58.6 | 56.1 | |
| PAC+Bubble (Randomized)Backbone=Flan-T5-L, Avg. Calls/Task=1842026.05 | 70.2 | 41 | 38.2 | 67 | 38.1 | 60 | 57.3 | 53.1 | |
| PAC+Bubble (Bidirectional)Backbone=Flan-T5-L, Avg. Calls/Task=3232026.05 | 69.3 | 44 | 41.4 | 68.5 | 39.2 | 61.7 | 57.2 | 54.5 | |
| BM252026.05 | 59.5 | 40.7 | 44.2 | 67.9 | 31.9 | 50.6 | 48 | 49 |