Improve Configuration Validation [ spike ]

[adamzimmermann]

Description

During drush deploy update hooks are ran and then configuration is imported.

This is great for deploying to production, but it makes it very hard to assert that the configuration in code matches the most up to date state of the database.

A module update done via one of our automated tools may have an update hook that changes configuration. The subsequent configuration import will likely blow those changes away, and then our configuration checking tool in Usher will not find an issue.

Questions / Areas of Uncertainty

• How can we check the state of configuration after database updates, but before configuration import?
• Do we need to override the drush deploy command or create or own version that extends it for CI usage?

Outcome

We can confidently validate that the configuration in code has the latest values, even if an automated dependency update includes changes to configuration via an update hook.

Additional Context

We will need a fully functioning/bootstrapped version of Drupal in a CI environment to validate this.

Existing Config Validation Code:

• https://github.com/ChromaticHQ/usher/blob/3.x/src/Robo/Plugin/Commands/ValidateConfigCommands.php#L79
• This only validates that the active and file configuration matches at the end of the whole process.

This might be worth exploring:

• https://mglaman.dev/blog/using-ddev-snapshots-speed-github-actions-workflows
• This repo might not be the best home for something like this though, as it is very opinionated.

@goranmiric and I had a good discussion about this recently somewhere, but I can't remember where. 🤔

Drupal Issues

• If an update hook modifies configuration, then old configuration is imported, the changes made by the update hook are forever lost. (directly related)
• Ensure that ConfigImport is taking place without outstanding updates (semi-related)
• CMI 2.0 Issues

Related Blog Posts

• https://chromatichq.com/insights/visualizing-drupals-configuration-management-process/ (general config theory)
• https://blog.dcycle.com/blog/2021-01-29/hook_update_n/ (update hook problems)

[markdorison]

A module update done via one of our automated tools may have an update hook that changes configuration. The subsequent configuration import will likely blow those changes away

If a new config item is added to Drupal core or contrib, and we run a config:import as part of drush deploy, I don't believe that config item would/should be overridden, unless something has changed in Drupal 10. Isn't this the scenario we created the "drupal-status" check to catch?

How can we check the state of configuration after database updates, but before configuration import?

Checking before config:import would fail (inaccurately) whenever we commit changes to a site's configuration.

[goranmiric]

@markdorison I just remembered one example.
The latest issue: Drupal 10.1 shipped with a new module phpass

• core hook_update enabled the module
• drush cim disabled it
  After that everything was in sync but actually we had an issue.

[markdorison]

@goranmiric Thanks for that example! That makes sense.

[adamzimmermann]

How can we check the state of configuration after database updates, but before configuration import?

Checking before config:import would fail (inaccurately) whenever we commit changes to a site's configuration.

Good point. 🙈

🤔 So we need to detect issues like the one @goranmiric described above, but not fail the more common scenario of us simply changing configuration files.

I need to ponder this.

[adamzimmermann]

So what we really need to do is:

1. run a config import
2. run update hooks
3. then check for a config diff

That does not match the order used for a production deployment or what drush deploy does, so wherever we run this would need to be isolated and be an additional process, not something we simply do alternatively in Tugboat or somewhere else.

That significantly adds to the overhead unless we can think of something clever. 😭

We almost need two databases to handle the two different states. 😬

---

• Alternatively could we check to see if the deployment will run any update hooks with updatedb:status
• If it will, just simply flag it some special way?
• Or grep the update hooks for some pattern or method call?
• Maybe just include a link to the diff in a GitHub comment?

[markdorison]

@adamzimmermann and I talked about this issue this morning and he has updated the issue description with links to some relevant Drupal core issues. I would also specifically note a comment I just made on this issue: If an update hook modifies configuration, then old configuration is imported, the changes made by the update hook are forever lost.

This issue recently affected us with the release of Drupal 10.1. #1845004: Replace custom password hashing library with PHP password_hash() included an update hook to enable the new phpass core module. Once that update hook ran, the module was enabled only to be disabled when config:import was subsequently run.

We have automated tests that check the status of Drupal config state, but these did not fail because once config:import had run (via drush deploy), the configuration on disk matched the active configuration as far as Drupal was concerned.

If we are allowing folks to believe that configuration on disk is their source of truth and then modifying active configuration with an update hook, this seems like it will inevitably cause issues like this.

[markdorison]

Moshe immediately responded in Drupal Slack:

[apotek]

A module update done via one of our automated tools may have an update hook that changes configuration.

This possibility should almost always/always line up with a change to at least mymodule/config/mymodule.schema.yml and possibly the config/install/mymodule.settings.yml.

So if there were a change in those files between versions, we could anticipate that there is an update hook that modifies config. I've always hated that drupal's update hooks are so powerful (you can do anything you want in there) and that they therefore can easily cause these circular dependencies on deployment. It really breaks the config files as the source of truth. To my mind, changes to the config/schema of a module should introduce a different major version number, and the update to the module should therefore include some extra steps and not have an automated update path.

But, back to reality.... I think the best we can do here is try to predict that there will be an issue and alert the end user that they should probably resolve config changes manually before deploying, and interrupt the deployment.