feat: Improvements/features around custom module images, registries and frontend builds

[EthanFreestone]

Custom module images, additional docker repos, and local frontend builds

Summary

This branch adds a handful of related features that make the CLI usable
against non-standard setups: modules hosted outside the usual registries,
extra docker repos, the ability to build and run a local, custom frontend
(e.g. a fork/branch of a platform repo) instead of always pulling the stock
platform image, and a self-healing persistent storage architecture that allows
local database configurations to cleanly survive application teardowns.

Includes some registry/http/build fixes that came up while getting the
above working reliably.

What's in here

Custom module image sources

Individual backend modules can now declare where their image actually
lives, instead of the CLI assuming a single default registry for
everything. Useful when you're running a fork of a module from a private
registry alongside the rest of the stock stack.

Additional docker repositories

The CLI can pull from more than one docker repo in a single run, with
fixes for cases where module-name matching was too eager and grabbing
the wrong module/version.

Per-module registry resolution

Registry lookups used to cycle through a shared, centralised list
("carousel") for every module. That's now resolved per-module instead,
which also fixed some hangs where a lookup would wait indefinitely on a
registry that didn't have the module.

HTTP snapshot lookup fix

Fixed a bug in the HTTP client where -SNAPSHOT version strings were
being mangled when translating between the URL-safe hyphenated form and
the real + separator used by the registry — this was causing failed
lookups for snapshot builds.

Local custom frontend builds

The main feature. You can now point the CLI at your own frontend repo
(fork, branch, whatever) and it will:

• clone it, build a stripes bundle from it in a multi-stage Dockerfile
  (build stage → nginx runtime stage), and
• deploy it as the tenant's UI container, instead of using the stock
  platform image.

Comes with:

• --ui-only flag to build/rebuild just the frontend image without
  touching the rest of the stack
• --no-cache flag to force a clean docker build
• an always-build config option, so you can opt a profile into
  rebuilding the frontend on every deploy rather than remembering
  --no-cache each time
• container port / memory fixes so the dev server actually has enough
  headroom to build and serve (Node needs more than the stock nginx
  container does)

Persistent volume preservation & self-healing gateway sync

Introduced robust data retention tools alongside automated orchestration network healing to optimize local developer loops:

• Data Volume Retention (--keepVolumes / -k Flag): Exposed the flag across undeployApplication, undeployModules, and undeploySystem. Passing -k forces container teardowns to skip volume wiping, allowing your Postgres tables, Keycloak configurations, and Kafka broker topics to persist locally.
• Ground-Truth Volume Checking (NewVolumes state flag): Bypassed fragile terminal log string matching. The CLI now dynamically runs an upfront docker volume inspect eureka_postgres_data command during deploySystem to determine whether this boot is a cold install or an environment restart using existing storage structures.
• Dynamic Gateway Cache Flushing (RefreshGateway state flag): Resolved a common container networking bug where a warm-volume boot caused Kong's internal upstream router states to become stale, throwing 503 name resolution failed errors on multi-tenant workflows. The runtime now flags warm boots and schedules an automated gateway synchronization flush (docker restart kong) to execute after all tracking modules are verified healthy, but before tenant initialization begins.
• disruption-Free Child Application Architecture: The gateway reset logic isolates itself exclusively to full, fresh standalone infrastructure executions. Layering secondary application tracks (like passing -p erm on top of an existing running environment) skips gateway cycles completely to protect your running connections.

Config-file support

Previously the Dockerfile always ran a fixed start script. That's swapped
out for a config-file option, so the build can point at any stripes
config file in the target repo, not just one hardcoded entry point. This
means the same mechanism works across different platform forks that use
different stripes config layouts.

Housekeeping

• centralised some duplicated module/UI/platform-tag resolution logic
  that had drifted across a few call sites
• registered frontend modules the same way backend modules already were
• bumped the kafka-tools image version
• reworked the custom-frontend Dockerfile's build stages to match the
  official platform-lsp nginx config more closely

Example configuration

Sample profile showing the new options in context (values are
illustrative, not the real ones from this repo):

profile:
  name: example
application:
  name: app-example
  version: 1.0.0

# Point individual modules at images hosted somewhere other than
# the default registry
backend-modules:
  mod-custom-thing:
    registry: "docker.example-org.com"
    image: "example-org/mod-custom-thing"
    private-port: 8080


# Build and deploy a local/forked frontend instead of the stock platform image
frontend:
  platform: platform-example
  url: https://gitlab.com/example-org/platform-example.git
  branch: feature/my-branch
  config-file: stripes.local-example.config.js   # any stripes config in the repo
  always-build: false   # set true to rebuild on every deploy;
                        # otherwise use `deploy/deployUi --build-ui --no-cache`
                        # for a one-off rebuild

Execution usage examples

Build just the frontend with a clean cache:

eureka-cli buildSystem --ui-only --no-cache

Tear down your application environment completely but preserve your underlying databases and tenant states:

eureka-cli -p ill undeploy -k

Redeploy over the retained configurations (the CLI will skip duplicate database creations and automatically re-heal the API gateway routing endpoints):

eureka-cli -p ill deploy

Testing

• Deploy a stack with a custom module image and confirm it pulls from the specified registry
• Deploy a stack pulling from two+ docker repos
• Build a local frontend with --ui-only, confirm it serves and hot-reloads as expected
• Confirm always-build: true triggers a rebuild on redeploy without needing --no-cache
• Confirm config-file correctly targets a non-default stripes config
• Run an explicit application teardown appending -k and verify your storage drives survive (Volume eureka_postgres_data Removing does not appear in console stdout logs)
• Deploy back over the kept storage data tracks and confirm that the engine skips duplicate asset generations but executes the post-readiness gateway sync step cleanly
• Add a layered child profile configuration mapping on top of a running environment and verify that it skips the gateway reset flow entirely

[alb3rtino]

Hi @EthanFreestone, I noticed this draft and wanted to flag two things early, while the design is still in motion:

Possible overlap with ECLI-26. ECLI-26 is in progress and covers running a FOLIO module that is not in the published registry, using a local source folder rather than an alternate registry. The use cases are complementary, but there is a direct design decision in common: ECLI-26 deliberately leaves the base application untouched and layers additional modules into an app-local child application, whereas InjectProfileModules adds missing profile modules to the module set used to create the active application. It might be worth talking to the implementing team and settling on an approach.

A possible split. This branch combines several changes with quite different review and risk profiles: registry/image overrides, missing-module injection, SNAPSHOT translation, custom frontend builds, volume-retention and gateway-recovery behavior, helper/refactoring work, documentation, and the kafka-tools bump. Splitting these would let the straightforward pieces land independently while the application-model, HTTP transport, frontend-build, and lifecycle changes could get focused review and testing.

[EthanFreestone]

@alb3rtino Thank you so much for reaching out :)

I definitely understand that this is a big chunk, and was unlikely to get in as is. I have very limited time I can afford to tooling, so my focus was mainly on "get the use cases working", namely:

• Need to be able to run a standalone eureka setup which uses some modules on different registries to the others
• Need to be able to keep volumes between deploys
• Need to be able to build a frontend platform image dynamically from a git repo

This turned out to need quite a lot of changes due to choices we'd made around versioning our images (the snapshot stuff) and specifics around hardcoded image namespaces in the CLI.

I wanted to make sure the community was aware of the changes I'd made on my fork, and to start discussions, but I'm not going to be able to invest enough time into getting the PR in shape to realistically make it in.

I do apologise for that, I know it's not particularly valuable work to drop a big proof of concept, but this unblocks us to test our modules in Eureka while we wait for official support of the use cases.

[EthanFreestone]

@pfilippov-epam I know was looking at these use cases at at one point, they might be interested in looking over these changes and taking them further, or might not, I'm not sure.

[alb3rtino]

Thanks @EthanFreestone, that is useful context. Given the scope, I think the best way forward would be to capture the three use cases as separate ECLI Jira tickets and link this PR as a proof of concept.

Each ticket should describe the concrete requirements and desired end-to-end workflow, with an example configuration, the commands involved, and the expected result. That would give development teams enough context to assess and prioritise the work.

[EthanFreestone]

@alb3rtino

I've spun up

https://folio-org.atlassian.net/browse/ECLI-34
https://folio-org.atlassian.net/browse/ECLI-35
https://folio-org.atlassian.net/browse/ECLI-36

as concepts, following the template from ECLI-26 as best as possible -- with the help of Claude Code.

I've left them in draft, because they are not necessarily fully fleshed out, although I have of course read through them, without this aid the tickets wouldn't get in at all.

The technical approaches, as with this ticket, are suggestions really.