Tabular Foundation Model Pretraining on TabArena 38 classification tasks subsampled 2025 (5-fold CV)
0.92Time (min)HPO, Muon WD, Mean-pool
Evaluation Results
| Method | Links | ||
|---|---|---|---|
| HPO, Muon WD, Mean-poolRecord #=9, Technique=HPO, Muon WD, Mean-pool2026.06 | 0.92 | 3,648 | |
| Repeated feature groupingRecord #=8, Technique=Repeated feature grouping2026.06 | 2.15 | 4,992 | |
| LAWA, AdamW WDRecord #=7, Technique=LAWA, AdamW WD2026.06 | 3.48 | 8,768 | |
| RMSNorm, Thinking RowsRecord #=6, Technique=RMSNorm, Thinking Rows2026.06 | 3.88 | 9,664 | |
| Residual decayRecord #=5, Technique=Residual decay2026.06 | 7.57 | 11,200 | |
| Batched Muon, compileRecord #=4, Technique=Batched Muon, compile2026.06 | 9.26 | 13,184 | |
| SDPA, bf16, LR, widthRecord #=3, Technique=SDPA, bf16, LR, width2026.06 | 10.1 | 13,184 | |
| Muon optimizerRecord #=2, Technique=Muon optimizer2026.06 | 54.41 | 45,824 | |
| BaselineRecord #=1, Technique=Baseline2026.06 | 74.32 | 80,576 |