Doom CLI Rewrite

[hlissner]

Important

This is a roadmap issue and is used solely to announce and document progress (to make it easier for users to follow and maintainers to track). Please do not use it for discussion; use our discussion board instead.

I've been working on this CLI rewrite for a while. Over time, bug reports and feature requests have trickled in for things that this rewrite is already slated to address, directly or indirectly. To reduce the noise I'm consolidating those and futures posts into this one, to represent the rewrite:

Primary goals of this rewrite

• (doom sync -u --force / doom upgrade --force discards local changes in package repos, making the "consume and contribute" straight.el workflow unviable #8838) "consumption and development" workflow: allow users to modify package source directly without it causing conflicts with Doom's package manager.

• (doom upgrade fails to upgrade org-mode and does return with exit code 0 #4270) To make our CLI transactional (failures should revert back to where we started rather than be ignored).
• ([REQUEST] Command to rollback upgrade #2141) To offer a doom rollback command (i.e. keep track of "generations" of doom syncs).
• (package! with pinning a specific commit does not update the commit in the straight repo directory. #3490) Make doom sync smarter about :recipe changes and stale bytecode (so you don't need to do that doom sync -u and doom build dance).
• Improve the doctor API and checks, so we get more useful checks and the warnings/errors they emit are less cryptic.
  • ([REQUEST] doom doctor to warn if using outdated module flags #2048) Check for misspelled modules and flags
  • (Need a warning when using package-install #4157) Warn users when they inadvertently use package.el
  • (doctor.el for doom-private-dir #4190) Support for a private doctor.el in your DOOMDIR
  • (Doctor should diagnose missing init.el, config.el, and packages.el and at least warn #2959) Perform more sanity checks on your private config
• To fix our unit tests and add integration tests, powered by nix and buttercup.
• CI/CD, so bumping packages can be automated (and tested).
• To handle straight.el's prompts better -- or prevent them entirely. Re-cloning packages is better UX than these confusing prompts.
• Add a new doom package ... family of commands for per-package management (e.g. doom package build PACKAGE will spare you an expensive doom build to rebuild all packages).
• Add a new doom module ... family of commands for toggling modules (or installing third party ones) programmatically (indirectly gives us a way to keep your init.el up-to-date across doom upgrades).
• Allow doom run to launch sandboxes and, if nix is available, do so from specific versions of Emacs.
• (Install script: Inline help for envvar file #4429, Doc: envar file #4880) Make doom install more beginner friendly
• (Introduce a CLI flag to perform non-interactive upgrade of plugins #4603) Add -f switch to doom sync to suppress prompts.
• ((package! :ignore t) doesn't seem to be working #4608, Use doom with straight #4823) Fix package!'s :ignore directive
• (Add "skip" option for uncommited changes during update #4712) Mark packages to be skipped when updating
• (local-repo autoloads created but never updated #4816) Autoloads aren't reindexed if package recipe contains :build (:not compile)
• (Doom compile should prompt to compile pdf-info-epdfinfo-program #5086) Build epdfinfo when pdf-tools package is installed/built.
• ("Doom + modules - private config" sandbox is still attempting to load private config from $DOOMDIR #5185) Isolate more in Doom's sandbox
• (Automatically (offer to) purge and rebuild nativecomp artifacts when comp-native-version-dir changes #5461) Have doom sync -p handle nativecomp artifacts.
• (Add a notifying line after native-compilation ends #5501) More informative/helpful output (e.g. that native compilation finished).
• (Bootstrapping doom with an literate configuration #6687) First class support for literate configs (allows init.el, too, to be generated).

This is not an exhaustive list.

Other bugs

• (doom sync & doom upgrade throw (wrong-type-argument arrayp nil) error on Windows #3844) Get /bin/sh on Windows to run chained scripts correctly
• ('doom sync' and 'doom upgrade' abort prematurely after "Restarting..." #3746) Fix bin/doom aborting unexpectedly when restarting itself
• (doom up breaks with "Failed to fetch from upstream", manual way works flawlessly #5362) Make doom upgrade atomic.
• (Remove "View the comparison diff in your browser?' confirmation on doom upgrade #5590) Prompt less on doom upgrade.

---

I'll keep this post updated until the rewrite is done. There is no ETA on this.

[danilevy1212]

Hi! Any way we can help you with this rewrite or contribute to its features? Thanks for the awesome work.

Add a new doom package ... family of commands for per-package management

I think it might be better to integrate this into existing commands. Using your example, instead of doom package build x, I think doom build x might be cleaner.

This is just a micro-aggression, but:

they use inadvertently use package.el

I'm pretty sure that's a typo

Also, I think it would be nice to put this in like a "cli" branch, considering how this could be a breaking change, but people might want to keep track of/try it on their own.