Ranking Preservation on SWE-bench Mini (test)
0.922Mean Spearman RhoEasiest
Evaluation Results
| Method | Links | ||||
|---|---|---|---|---|---|
| Easiestk=20, Selection strategy=Easiest, Evaluation protocol=mean across all protocols2026.03 | 0.922 | — | — | — | |
| MRk=20, Selection strategy=Mid-Range, Evaluation protocol=mean across all protocols2026.03 | 0.92 | — | — | — | |
| Randomk=20, Selection strategy=Random, Evaluation protocol=mean across all protocols2026.03 | 0.903 | 0.026 | 0.695 | 0.977 | |
| Stratifiedk=20, Selection strategy=Stratified, Evaluation protocol=mean across all protocols2026.03 | 0.902 | — | — | — | |
| Greedyk=20, Selection strategy=Greedy, Evaluation protocol=mean across all protocols2026.03 | 0.881 | — | — | — |