Skip to content

fix: env/ group access and e2e fixture names - #145

Merged
catinspace-au merged 1 commit into
mainfrom
fix/env-group-and-e2e-names
Sep 26, 2026
Merged

catinspace-au merged 1 commit into
mainfrom
fix/env-group-and-e2e-names

Conversation

@catinspace-au

Copy link
Copy Markdown
Contributor

Two small rc.14 fixes in one PR.

First, env/ is shared with the engine through the checkout's group. The engine writes env/<app>.custom.env into the operator's checkout as UID 1000. On a host where the operator is anyone else, the engine could not write into env/, and Compose could not read the 0600 file it wrote.

  • dfe-engine gets group_add: ["${DFE_DEV_GID:-1000}"]. The Makefile already exports DFE_DEV_GID as id -g.
  • make init and every start target (via env_files.py) make env/ group-writable and setgid, so each new file keeps the checkout's group. It only ever adds bits. If it cannot chmod, it warns with the command to run.
  • The engine half (write 0640, take the directory's group, warn with the fix when it cannot) is fix: write each app's custom env file readable by its directory's group dfe-engine#545.

Second, two e2e fixtures are renamed off the names the engine generates. dfe-transform-vector-filebeat and dfe-transform-elastic-cisco-ios are exactly what the engine calls a real instance for a source named filebeat or cisco-ios. Compose merges two services sharing a name, so a deployment with either source would silently take the fixture's command.

  • Now dfe-transform-e2e-vector-filebeat and dfe-transform-e2e-elastic-cisco-ios, the same shape test(e2e): kill hops under load and count the loss #143 gives the VRL one.
  • Swept: compose (base, override, container-logs), service_profiles.yaml, resolve_profile, check_compose, build_dev_images, test_e2e, the schema-control test, docs, and both env templates (git mv).
  • post.py: the vector fixture no longer starts with dfe-transform-vector, so it is named in full in OTEL_PUSHER_PREFIXES.
  • The DFE_TRANSFORM_VECTOR_FILEBEAT_* and DFE_TRANSFORM_ELASTIC_CISCO_IOS_* variables keep their names, so an existing .env still applies. The Kafka group_id in config/transform-elastic/cisco-ios.yaml is left alone. It models the real instance's group and is not a compose name.

HEADS UP on #143: it also adds the same group_add hunk (identical text and position, so that part merges clean) and its own grant_group_write() in env_files.py without setgid. Whichever lands second has a small conflict in env_files.py and in the lists both PRs re-sort. Keep share_env_dir over grant_group_write, because setgid is the part that hands new files the group.

make check: 307 passed. New tests in scripts/tests/test_env_dir.py, and three of them fail with the chmod disabled. Not run: a real stack as a non-1000 operator.

The engine writes env/<app>.custom.env into the operator's checkout as UID 1000. On a host where the operator is anyone else it could not write there, and Compose could not read what it wrote. dfe-engine now runs with the checkout's group (group_add, DFE_DEV_GID), and make init plus every start target make env/ group-writable and setgid, so each new file keeps the group Compose reads it as.

dfe-transform-vector-filebeat and dfe-transform-elastic-cisco-ios were exactly the names the engine gives a real instance for a source called filebeat or cisco-ios, and Compose merges two services sharing a name. They are now dfe-transform-e2e-vector-filebeat and dfe-transform-e2e-elastic-cisco-ios, with their env templates renamed to match. The DFE_TRANSFORM_*_FILEBEAT and *_CISCO_IOS variables keep their names so an existing .env still applies.
@catinspace-au
catinspace-au merged commit 445d454 into main Sep 26, 2026
4 checks passed
@catinspace-au
catinspace-au deleted the fix/env-group-and-e2e-names branch September 26, 2026 11:52
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant