Skip to content

Opimizer issue (found loss = NaN in second fold) #1

Description

@xydxdy

Due to TensorFlow's compilation issues, the second loop attempts to use the first initialize object of the optimizer in the __init__ function as the loss return NaN value (Even if the new model object is declared within the loop, the first initialize optimizer is attempted)

To resolve this issue, do not utilize the optimizer's initial value as provided by the __init__ function; instead, the optimizer must be assigned in each and every fold as in the example below:

for fold in range(1, 6):
    # the optimizer must be assigned in each and every fold
    optimizer = Adam(beta_1=0.9, beta_2=0.999, epsilon=1e-08)
    model = MIN2Net(optimizer =optimizer)

However, this issue does not affect the experimental results of our paper since we declared a new optimizer in every loop https://github.com/IoBT-VISTEC/MIN2Net/blob/main/experiments/train_MIN2Net_k-fold-CV.py#L41

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions