net.zero_grad()

[jason-engage]

Hi, I'd like to try this in my u2net imagenet model.

Can i simply drop it in and replace Adam/SGD?

I noticed that you are calling net.zero_grad() in your training loop, however in my script its optimizer.zero_grad()... will this still work with AGD set as the optimizer?

Any other suggestions on getting it to work correctly? Thanks for putting another idea out there, Cheers

[jxbz]

Hi Jason, it may or may not work just yet as a simple drop-in. For instance, we still don't support all layer types. That said, optimizer.zero_grad() should work now since we subclass the PyTorch optimizer class. We've been continuing to work on the underlying science rather than prioritising making something that's easy to use, but hopefully that will come at some point.