In the forward of class WideResNet，there is a BN normalizing the output of the last res-block and if I comment this line, the loss in the model will become NaN after few steps (in a high learining rate). I want to ask whether the case just appear in MPL or I should note it in other algorithm?
In the forward of class WideResNet，there is a BN normalizing the output of the last res-block and if I comment this line, the loss in the model will become NaN after few steps (in a high learining rate). I want to ask whether the case just appear in MPL or I should note it in other algorithm?