Skip to content

Possible unintended change in convergence behavior #96

Description

@michaelmckinsey1

We overlooked a change when reviewing #88 to convergence behvaior. The validation dice calculation changed to argmax instead of softmax (F05 from the REVIEW.md document).

Argmax removes the constraint for the model to improve prediction confidence beyond majority. For example, how the validation dice is measured now vs originally for two cases (both background majority prediction):

CURRENT (#88/#92, hard val dice)
  background: 0.51, foreground: 0.49
  train loss: 1.673, val loss: 1.673, train dice: 0.000002, val dice: 1.000000

  CURRENT (#88/#92, hard val dice)
  background: 0.99, foreground: 0.01
  train loss: 1.010, val loss: 1.010, train dice: 0.000100, val dice: 1.000000

  ORIGINAL (soft val dice)
  background: 0.51, foreground: 0.49
  train loss: 1.673, val loss: 1.673, train dice: 0.000002, val dice: 0.000002

  ORIGINAL (soft val dice)
  background: 0.99, foreground: 0.01
  train loss: 1.010, val loss: 1.010, train dice: 0.000100, val dice: 0.000100

(train loss, val loss, and train dice all softmax still).

We must decide which metric we want to use, as using argmax or softmax have different meanings for this benchmark. Argmax will obviously converge faster than softmax.

Image Image

Metadata

Metadata

Labels

No labels
No labels

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions