Skip to content

About

No description, website, or topics provided.

Resources

Stars

12 stars

Watchers

0 watching

Forks

Latest commit

 

History

5 Commits

Folders and files

NameName
Last commit message
Last commit date
 
 
 
 
 
 
 
 
 
 

Repository files navigation

In Search of Adam’s Secret Sauce

Antonio Orvieto & Robert Gower

With this repository we share

  • Our extensive experimental data comparing optimizers on language models (data folder). Note that some details might be missing, you can find everything in the paper
  • Our visualizations scripts, so you can better match with the paper (tuning plots folder)
  • We additionally share all training curves!
  • Our quadratic example folder serves as a playground to understand the Adam/SGD/Signum gap in language models!

About

No description, website, or topics provided.

Resources

Stars

12 stars

Watchers

0 watching

Forks

Releases

Packages

Contributors

Languages