Fix bug where Nout=1, leading to erroneous predictions

[xbyrne]

While applying SPOCK within an MCMC, I found that a large region of parameter space had the exact same stability probability of 0.8972374 . After some digging, it turned out that this was because Nout=1 in this region, presumably due to a very short secular timescale. As a result, some features were erroneously calculated to be 0 or NaN, giving a mode collapse in the stability probability estimation.

This was amended by setting Nout to be at least 3, which allowed for more robust feature calculation and probability estimation.

The diff is highly simple and intuitive, and can be automatically merged.

[gitnotebooks]

Review these changes at https://app.gitnotebooks.com/dtamayo/spock/pull/51

[dtamayo]

Hi Xander,

Nice catch! We've also run into weird edge cases when running big sets of systems across a wide range of parameter space, and have been trying to patch them.

I guess if the secular timescale is so short that Nout <= 1, it means that the system isn't behaving secularly at all. I wonder whether it might make sense to set a floor of something like Norbits = 1000 rather than a floor on Nout? Doing that would likely provide enough integration time for Nbody to catch the instability--I certainly wouldn't trust the outputs of the model that far from the training set conditions (with 3 outputs across a very short Nbody integration).

What do you think?

[xbyrne]

Hi Dan,

That makes a lot of sense. A floor of Norbits=1000 would be equivalent to a floor of Nout=8, which would of course be better than Nout=3; and Norbits is semantically a better choice of variable to floor than Nout.

Either way, the cases where Nout=3 for me seemed to be flagged as unstable anyway; though maybe that was specific to my case :)