Checkpoint Ranking on Group II Checkpoints at different pre-training stages
100Recall@1NLEEP
Evaluation results
| Method | Links | ||||||
|---|---|---|---|---|---|---|---|
| NLEEPGFLOPS=12.952020.11 | 100 | 100 | 100 | 100 | 72.84 | 67.49 | |
| Linear (converged)GFLOPS=5.33E42020.11 | 75 | 99.95 | 100 | 100 | 79.3 | 68.6 | |
| LEEPGFLOPS=378.312020.11 | 75 | 99.44 | 75 | 99.9 | 50.36 | 55.49 | |
| Linear (5 epochs)GFLOPS=4.97E42020.11 | 50 | 99.57 | 100 | 100 | 55.07 | 51.28 | |
| MI w/ PCAalpha=0.01, GFLOPS=5.58E42020.11 | 50 | 99.6 | 75 | 99.85 | 52.14 | 51.34 | |
| Fine-tune (1 epoch)GFLOPS=6.51E52020.11 | 25 | 99.05 | 25 | 99.47 | 19.61 | 15.52 | |
| Fine-tune (5 epochs)GFLOPS=4.28E62020.11 | 25 | 99.55 | 100 | 100 | 68.47 | 58.33 | |
| Linear (1 epoch)GFLOPS=4.95E42020.11 | 0 | 96.46 | 25 | 98.79 | 27.01 | 24.24 | |
| MIalpha=0.01, GFLOPS=1.62E52020.11 | 0 | 94.84 | 25 | 97.43 | -29.41 | -17.81 | |
| MIalpha=0.5, GFLOPS=1.62E52020.11 | 0 | 96.66 | 0 | 97.03 | -11.36 | -10.21 | |
| MI w/ PCAalpha=0.5, GFLOPS=5.58E42020.11 | 0 | 96.68 | 50 | 99.52 | 23.73 | 17.09 |