loss mean instead of dividing by loss mask.sum()

[Manjunath679]

in traineagle3 loss is calculate as mean for all positions ,but in train loss is divided by loss_mask.sum() , why is such?

[AchuthReddy-16]

I looked into this a bit and noticed a small difference in how the loss is normalized between the original training code and EAGLE-3.

In the original EAGLE-1/2 training code, the loss is divided by loss_mask.sum(), so it is normalized only over the valid/unmasked tokens.

But in eagle/traineagle3/cnets.py, the loss is computed with .mean():

loss = -torch.sum(position_mask * plogp, 2).mean()

This averages over all batch/sequence positions, including positions where the mask is zero. Because of that, examples with more masked-out positions may contribute less to the loss compared to the original normalization.

Was this change intentional for EAGLE-3’s training setup, or should it be normalized by position_mask.sum() similar to the earlier training code?