Ensure PSD-safe factorization in constructor of MultivariateNormal

[SebastianAment]

There was a recent BoTorch issue that was caused by a positive semi-definite matrix being passed to MultivariateNormal as a Tensor, which causes the constructor to fail because PyTorch's constructor calls cholesky on the tensor. This commit upstreams the corresponding BoTorch PR to ensure that all covariance matrices are LinearOperator types, thereby triggering _psd_safe_cholesky, whenever cholesky is called.

[Balandat]

Thanks! An alternative to making everything into a LinearOperator would be upon receiving a dense tensor, rather than just storing that and passing it along, to compute the cholesky decomposition with jitted and then pass that as the scale_tril to the torch distribution. The downside of that is ofc that this would do a lot of compute upon construction of the object, potentially unnecessary. Another hack could be to mock some of the torch distribution code so that it users the psd-safe cholesky decomposition internally (though that seems very hacky and potentially problematic).

[Balandat]

It seems odd to have _islazy set to True if the covariance matrix is indeed a LinearOperator. I guess the "lazy" nomenclature is a bit outdated anyway with the move to LinearOperator.

[Balandat]

Do you mean changing the torch code to validate LinearOperator inputs? That might be somewhat challenging to do if we want to use LinearOperators there explicitly. What would work is to make changes in pure torch that would make it easier to use LinearOperator objects by means of the __torch_function__ interface we define in LinearOperator.