Very niche problem, when using multi-discrete, and apparently only while running without a GPU, at some point a NaN pops up in probabilities. I caught in ppo_learner on line 90, old_probs had a single NaN, if left to go another cycle, all policy weights become NaNs. No non-finite values caught in obs to cause this.
Very niche problem, when using multi-discrete, and apparently only while running without a GPU, at some point a NaN pops up in probabilities. I caught in ppo_learner on line 90, old_probs had a single NaN, if left to go another cycle, all policy weights become NaNs. No non-finite values caught in obs to cause this.