CTCMWER Loss may be negative sometime ?

[sheirving]

Thankyou very much for share the code, but when I training a model using the CTCMWER loss, it may be negative, sometime? So what's the reason could be?

[TeaPoly]

Thankyou very much for share the code, but when I training a model using the CTCMWER loss, it may be negative, sometime? So what's the reason could be?

CTC-OptimizedLoss/mwer_loss.py

Line 75 in c5b0c17

# Re-normalized error word number over just the N-best hypotheses

It maybe negative when re-normalized error word number.

[zhhuang93]

Hi, thanks for sharing. Is it normal for a negative loss?

[TeaPoly]

Hi, thanks for sharing. Is it normal for a negative loss?

It is normal to observe that the loss value is getting smaller and smaller, because the average word error is subtracted when normalizing. For details, please refer to the following formula from paper https://arxiv.org/abs/1712.01818.
In addition, don't use MWER loss in the initial stage of training, and try to use CTC loss for initial training.