AdamW Is Dead for Tabular MLPs: Lion and Sophia Win the Benchmark
A rigorous benchmark of 19 optimizers on 45 tabular datasets shows that Lion and Sophia beat AdamW, the default optimizer for tabular MLPs. This paper tells the field: your go-to choice is leaving performance on the table.











