Skip to content

Direct dynamic setting or change of ccov #349

@nikohansen

Description

@nikohansen

Similar to #346, we want to be able to set or change "$c_\text{cov}$".

Review possible ways:

  • make option 'CMA_on' (and 'CMA_rankone' and 'CMA_rankmu') versatile to be read in dynamically. They are already used to (re-)set the learning rate before the update.
  • call weights.finalize after self.sm.parameters(), i.e. after resetting the learning rate, (possibly conditioned to any of the multipliers have change)
  • The "correct" way to change $c_\text{cov}$, say by $\alpha$, is by re-computing $c_1, c_\mu, c_c$ from $\alpha\mu_w$ where $\alpha < 1 / \mu_w$ would need to be checked carefully?
  • ...

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions