Training Blows up for 8 Gaussians

[jiachangliu]

Hi Jarrel,

I'm trying to adapt your code to fit 8 Gaussian mixtures like what experiment they showed in their paper (unlike you fitting the latent z with 4 Gaussian mixtures, I'm trying to fit the latent z with a Gaussian prior). However, loss, logdet, and pz all blow up (na) after some point.

Could you guess what the problem is and how to make the training more stable? Thanks.

[jarrelscy]

Hi Jiachang - what batch size are you using? It may be related to learning rate / batch size. Also, the authors have released their official code at https://github.com/jhjacobsen/invertible-resnet - their implementation is very similar to mine but perhaps give theirs a go and see if it doesn't blow up?

…

On Fri, Jul 26, 2019 at 3:22 PM Jiachang Liu ***@***.***> wrote: Hi Jarrel, I'm trying to adapt your code to fit 8 Gaussian mixtures like what experiment they showed in their paper (unlike you fitting the latent z with 4 Gaussian mixtures, I'm trying to fit the latent z with a Gaussian prior). However, loss, logdet, and pz all blow up (na) after some point. Could you guess what the problem is and how to make the training more stable? Thanks. — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#4?email_source=notifications&email_token=AAOEUB3KYLI7KESBS722LS3QBKCYDA5CNFSM4IHARWO2YY3PNVWWK3TUL52HS4DFUVEXG43VMWVGG33NNVSW45C7NFSM4HBUF6MQ>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAOEUB4CDLHU5XLX6GQWPQ3QBKCYDANCNFSM4IHARWOQ> .