ResearchBenchmarksOptimizer Memory Footprint Analysis on MNIST (train)Optimizer Memory Footprint Analysis on MNIST (train)Follow0.27Peak Memory UsageAdam4bit0.15880.90941.662.4106Jun 11, 2026Evaluation resultsMethodMethodLinksPeak Memory UsagePersistent Memory UsageAdam4bit2026.060.270.27Gefen2026.060.290.27Adam8bit2026.060.510.51AdamW2026.0622SM32026.062.021.01Adam-mini2026.062.971Adafactor2026.063.051.02