Structural retrieval on SCOPe-40
50.7Recall@1ProtSent
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| ProtSentModel Parameters=150M2026.05 | 50.7 | 68.5 | 72.4 | — | |
| ProtSentModel Parameters=35M2026.05 | 44.5 | 65.1 | 71 | — | |
| ESM2-150M Masked-pos (run 1)Backbone=ESM2-150M, Objective=Masked-pos, Seeds=42/123/456, Training budget=8 h2026.05 | 42.7 | — | — | — | |
| ESM2-150M Masked-pos (run 2)Backbone=ESM2-150M, Objective=Masked-pos, Seed=42, Training budget=8 h2026.05 | 42.5 | — | — | — | |
| ESM2-150M BaselineBackbone=ESM2-150M, Objective=Baseline, Training budget=8 h2026.05 | 42.4 | — | — | — | |
| ESM-2Model Parameters=150M, Status=baseline2026.05 | 42.3 | 58.9 | 64.4 | — | |
| ESM2-150M EMA (4h)Backbone=ESM2-150M, Objective=EMA, Training budget=4 h2026.05 | 42.3 | — | — | — | |
| ESM2-35M Masked-pos (AdamW)Backbone=ESM2-35M, Objective=Masked-pos (AdamW), Training budget=8 h2026.05 | 39.9 | — | — | — | |
| ESM2-150M All-pos MLM+JEPABackbone=ESM2-150M, Objective=All-pos MLM+JEPA, Training budget=8 h2026.05 | 39 | — | — | — | |
| ESM-2Model Parameters=35M, Status=baseline2026.05 | 38.5 | 58.8 | 64.1 | — | |
| ESM2-35M BaselineBackbone=ESM2-35M, Objective=Baseline, Training budget=8 h2026.05 | 38.2 | — | — | — | |
| ESM2-35M Masked-pos (SGDm)Backbone=ESM2-35M, Objective=Masked-pos (SGDm), Training budget=8 h2026.05 | 36.8 | — | — | — | |
| ESM2-150M MLM-onlyBackbone=ESM2-150M, Objective=MLM-only, Training budget=8 h2026.05 | 34.5 | — | — | — | |
| ESM2-35M MLM-onlyBackbone=ESM2-35M, Objective=MLM-only, Training budget=8 h2026.05 | 33.9 | — | — | — | |
| ESM2-35M All-pos MLM+JEPABackbone=ESM2-35M, Objective=All-pos MLM+JEPA, Training budget=8 h2026.05 | 33.2 | — | — | — | |
| ProteinBERT2-35M Masked-posBackbone=ProteinBERT2-35M, Objective=Masked-pos, Training budget=8 h2026.05 | 21.1 | — | — | — | |
| AMPLIFY-120M All-pos MLM+JEPABackbone=AMPLIFY-120M, Objective=All-pos MLM+JEPA, Training budget=8 h2026.05 | 21 | — | — | — | |
| Rand-Init-35M All-pos MLM+JEPABackbone=Rand-Init-35M, Objective=All-pos MLM+JEPA, Training budget=8 h2026.05 | 20.6 | — | — | — | |
| Rand-Init-35M MLM-onlyBackbone=Rand-Init-35M, Objective=MLM-only, Training budget=8 h2026.05 | 19.4 | — | — | — | |
| ProteinBERT2-150M masked-posBackbone=ProteinBERT2-150M, Objective=masked-pos, Training budget=8 h2026.05 | 19.1 | — | — | — | |
| AMPLIFY-120M Masked-posBackbone=AMPLIFY-120M, Objective=Masked-pos, Training budget=8 h2026.05 | 18.4 | — | — | — | |
| Rand-Init-35M Masked-posBackbone=Rand-Init-35M, Objective=Masked-pos, Training budget=8 h2026.05 | 18.2 | — | — | — | |
| AMPLIFY-120M MLM-onlyBackbone=AMPLIFY-120M, Objective=MLM-only, Training budget=8 h2026.05 | 16.7 | — | — | — | |
| AMPLIFY-120M BaselineBackbone=AMPLIFY-120M, Objective=Baseline, Training budget=8 h2026.05 | 15.1 | — | — | — | |
| ESM2-35M JEPA-onlyBackbone=ESM2-35M, Objective=JEPA-only, Training budget=8 h2026.05 | 13.9 | — | — | — | |
| ProteinBERT2-35M MLM-onlyBackbone=ProteinBERT2-35M, Objective=MLM-only, Training budget=8 h2026.05 | 8.5 | — | — | — | |
| ProteinBERT2-35M All-pos MLM+JEPABackbone=ProteinBERT2-35M, Objective=All-pos MLM+JEPA, Training budget=8 h2026.05 | 8.4 | — | — | — | |
| ProteinBERT2-35M BaselineBackbone=ProteinBERT2-35M, Objective=Baseline, Training budget=8 h2026.05 | 8.2 | — | — | — | |
| ProteinBERT2-35M JEPA-onlyBackbone=ProteinBERT2-35M, Objective=JEPA-only, Training budget=8 h2026.05 | 8 | — | — | — | |
| Rand-Init-35M BaselineBackbone=Rand-Init-35M, Objective=Baseline, Training budget=8 h2026.05 | 7.9 | — | — | — | |
| Rand-Init-35M JEPA-onlyBackbone=Rand-Init-35M, Objective=JEPA-only, Training budget=8 h2026.05 | 5.4 | — | — | — | |
| ESM2-150M JEPA-onlyBackbone=ESM2-150M, Objective=JEPA-only, Training budget=8 h2026.05 | 3.7 | — | — | — | |
| ESM2-150M rand-init MLM-onlyBackbone=ESM2-150M, Objective=rand-init MLM-only, Training budget=8 h2026.05 | 3.4 | — | — | — | |
| AMPLIFY-120M JEPA-onlyBackbone=AMPLIFY-120M, Objective=JEPA-only, Training budget=8 h2026.05 | 0.4 | — | — | — |