research
∙
04/27/2023
Noise Is Not the Main Factor Behind the Gap Between SGD and Adam on Transformers, but Sign Descent Might Be
The success of the Adam optimizer on a wide array of architectures has m...
share
research
∙
12/01/2021
