Adding Bernoulli Distribution

[SabrinaRichter]

No description provided.

[SabrinaRichter]

recent status of gradients: tensorflow-gradient seems to be wrong as gradients for the coefficients that are initialized with zeros (and should thus be trained) are zero. Computation of log_probs wrt model_loc and model_scale seems to be correct and the only things happening before that are eta_loc = tf.matmul(design_loc, a_var) and model_loc = 1/1+tf.exp(-eta_loc) and the same for eta_scale.

Will look at it again during the next days.

[davidsebfischer]

Did you try initializing at non-zero? Or checked whether zero is is an additional extremum?

[SabrinaRichter]

Status report:

I think the hessian test didn't do what it was supposed to do and fixed it.

Bernoulli should be more or less ok:
Tests for graphs, jacobians, hessians pass. Also acc_analytic, when not sparse. Trying other optimizers (e.g. nr) throws errors, but also for nb. Don't know wether you fixed that yesterday.

I also compared the old beta version to nb, but I didn't find anything that could solve the gradient problem. The only thing I am not sure about are the bounds. Maybe you can have a look at them?

The new version for beta is not yet completely working:
Jacobians are ok, hessians are definitely "the same" in the test, but have relatively high differences.
Closed_form initialization can't handle different design matrices (yet).
(Standard-)Initialization could be ok by now, but training does definitely not work yet. (I will work on that next week)
I also noticed, that the off-diagonal-part of the fim is here non-zero. I am not sure if this is already compatible with your implementation of irls.

(btw: beta2 is the old version with mean and samplesize and beta is the new version with p and q)

[davidsebfischer]

Ok, nice! I will add an option to use the full FIM as oppose to the block-diagonal blocks only. Until then, just set fim_a and fim_b as zero scalars and return a 3rd object "fim".

[SabrinaRichter]

Forgot log in the log-likelihood, so no fim at all. Sorry!