Optimization Efficiency on SST-2 full (train)
1,003StepsB-VN-v2
Evaluation Results
| Method | Links | |
|---|---|---|
| B-VN-v2EMA smoothing factor=0.98, Target loss threshold=0.05, Weight decay=None, Algorithm=BregTop, Preconditioner=VN, Variant=v22026.05 | 1,003 | |
| B-VN-v1EMA smoothing factor=0.98, Target loss threshold=0.05, Weight decay=None, Algorithm=BregTop, Preconditioner=VN, Variant=v12026.05 | 1,018 | |
| B-F-v2EMA smoothing factor=0.98, Target loss threshold=0.05, Weight decay=None, Algorithm=BregTop, Preconditioner=F, Variant=v22026.05 | 1,059 | |
| B-F-v1EMA smoothing factor=0.98, Target loss threshold=0.05, Weight decay=None, Algorithm=BregTop, Preconditioner=F, Variant=v12026.05 | 1,084 | |
| KLEMA smoothing factor=0.98, Target loss threshold=0.05, Weight decay=None2026.05 | 1,092 | |
| ShampooEMA smoothing factor=0.98, Target loss threshold=0.05, Weight decay=None2026.05 | 1,119 | |
| VN-v2EMA smoothing factor=0.98, Target loss threshold=0.05, Weight decay=None, Variant=v22026.05 | 1,127 | |
| VN-v1EMA smoothing factor=0.98, Target loss threshold=0.05, Weight decay=None, Variant=v12026.05 | 1,141 | |
| F-v1EMA smoothing factor=0.98, Target loss threshold=0.05, Weight decay=None, Variant=v12026.05 | 1,598 | |
| F-v2EMA smoothing factor=0.98, Target loss threshold=0.05, Weight decay=None, Variant=v22026.05 | 1,602 |