Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
278 commits
Select commit Hold shift + click to select a range
6f55959
fix: POST passes on a Kafka stack from zero (#153)
catinspace-au Sep 27, 2026
6fb8648
fix: source and flow tests run on a default .env (#154)
catinspace-au Sep 27, 2026
6893f53
fix: add the make apply target the engine hints name (#156)
catinspace-au Sep 27, 2026
4e2879f
fix(compose): one prefix runs a second stack (#128)
catinspace-au Sep 27, 2026
276c9be
fix(compose): dial ClickHouse on its container ports (#161)
catinspace-au Sep 27, 2026
55df4ae
fix(post): wait out the loader's topic refresh on the routing claim (…
catinspace-au Sep 27, 2026
800646a
fix(ci): run the helper scripts on Python 3.14 (#164)
catinspace-au Sep 27, 2026
b9efeda
fix(compose): HyperDX dials an external ClickHouse (#163)
catinspace-au Sep 27, 2026
9159302
fix(post): schema wait reads the engine in-network (#165)
catinspace-au Sep 27, 2026
0a43c6c
fix(config): start vrl and engine-bound vector groups at the log star…
catinspace-au Sep 27, 2026
5d5aed7
fix(apply): remove a deleted source's container, and read .env only w…
catinspace-au Sep 27, 2026
507342d
fix(apply): a bare apply removes deleted sources, and the committed v…
catinspace-au Sep 27, 2026
282cc88
fix(compose): read the custom env files the engine writes (#169)
catinspace-au Sep 27, 2026
941f489
docs: a source's fetcher and transform run one instance per source on…
catinspace-au Sep 27, 2026
47e6529
fix(init): .env is never readable by other users (#171)
catinspace-au Sep 28, 2026
d8ee7d1
fix: give the engine this stack's admin UI links (#172)
catinspace-au Sep 28, 2026
d921fff
fix(tests): no test run deletes the checkout's generated compose frag…
catinspace-au Sep 28, 2026
ce2a1b0
fix: name what overrides the receiver's loader (#174)
catinspace-au Sep 28, 2026
b69fe76
fix: push dfe-ui's traces to the collector's HTTP port (#175)
catinspace-au Sep 28, 2026
c81294e
fix(e2e): prove rules and hunts on the stack (#176)
catinspace-au Sep 28, 2026
842b90f
fix(post): hunt claim uses run-now and exact match (#177)
catinspace-au Sep 28, 2026
f568f83
fix: pin the container prefix in tests and the POST claim count (#178)
catinspace-au Sep 28, 2026
8ba558f
fix: point the engine at each app's metric manifest (#180)
catinspace-au Sep 29, 2026
411769b
fix(e2e): prove Search to rule to hunt in the console (#179)
catinspace-au Sep 29, 2026
96125ea
fix: re-render service profiles from dfe-infra apps.yaml (#181)
catinspace-au Sep 29, 2026
289c86c
fix(fetcher): keep cursors across restarts
Sep 29, 2026
d3c9c41
Merge pull request #183 from hyperi-io/fix/182-fetcher-cursor-dir
kazmosahebi Sep 29, 2026
404f62f
fix(clickhouse): bound the server log and the system log tables (#184)
catinspace-au Sep 29, 2026
2ea5d07
fix: set a credential file's mode before its text lands (#186)
catinspace-au Sep 29, 2026
7ad96f4
fix: give hyperdx its own memory limit (#185)
catinspace-au Sep 29, 2026
084c6dd
fix(e2e): stop the search test clearing its own source (#187)
catinspace-au Sep 29, 2026
879a4ec
fix: turn hyperdx usage stats off by default (#188)
catinspace-au Sep 29, 2026
eecb44b
fix: build make dev images the way the release builds them (#189)
catinspace-au Sep 29, 2026
d63a743
fix: say what seeds a HyperDX team, not the engine's service token (#…
catinspace-au Sep 29, 2026
823fc28
fix: show hyperdx as Search in the compose admin links (#191)
catinspace-au Sep 29, 2026
cab0827
fix: resolve the three high code-scanning alerts in the helper script…
catinspace-au Sep 30, 2026
c7fd415
test(e2e): prove a SIGKILLed hop loses nothing (#195)
catinspace-au Oct 1, 2026
fda20cb
fix(stack): pin the stack dial at 2.2.0-rc.14 (#196)
hypersec-ci-bot[bot] Oct 4, 2026
41d1aa3
fix(post): send current_password when replacing an issued password (#…
catinspace-au Oct 4, 2026
cd65212
fix: keep the dead-letter queue on in the shipped loader configs (#199)
catinspace-au Oct 4, 2026
44de347
chore(deps): Update actions/checkout action to v7.0.1 (#200)
renovate[bot] Oct 5, 2026
9425d37
fix: drop the metrics manifest URL the engine no longer reads (#203)
catinspace-au Oct 5, 2026
d5a0ef7
fix: install ruamel.yaml for the profile check (#205)
catinspace-au Oct 6, 2026
f940665
fix: mint a per-deploy HyperDX session secret (#204)
catinspace-au Oct 6, 2026
7970385
fix(deps): bump oauth2-proxy to v7.15.5 for two critical auth bypasse…
catinspace-au Oct 6, 2026
2105701
fix: the hunt runner gets its own ClickHouse user (#197)
catinspace-au Oct 6, 2026
37a8817
fix: make a VM update survive its first start (#209)
catinspace-au Oct 6, 2026
be9426f
fix(stack): pin the stack dial at 2.2.0 (#208)
hypersec-ci-bot[bot] Oct 6, 2026
9554346
fix(proxy): opt-in console TLS so OIDC works
kazmosahebi Oct 6, 2026
3ee222f
Merge pull request #210 from hyperi-io/fix/console-tls
kazmosahebi Oct 6, 2026
72d4299
fix(stack): pin the stack dial at 2.2.1-rc.1 (#214)
hypersec-ci-bot[bot] Oct 10, 2026
852b7c3
fix: build Rust components with 1.99, the toolchain CI uses (#215)
catinspace-au Oct 10, 2026
03ee6b9
fix: run the e2e suites under console TLS (#216)
catinspace-au Oct 10, 2026
0f2cb6e
fix(stack): pin the stack dial at 2.2.1-rc.2 (#217)
hypersec-ci-bot[bot] Oct 10, 2026
5ea618b
fix(stack): pin the stack dial at 2.2.1-rc.3 (#218)
hypersec-ci-bot[bot] Oct 10, 2026
a202782
docs: Google groups come from the user's own token, not a service acc…
catinspace-au Oct 10, 2026
aee5c8d
fix(stack): pin the stack dial at 2.2.1-rc.4 (#220)
hypersec-ci-bot[bot] Oct 10, 2026
b117341
fix(stack): pin the stack dial at 2.2.1-rc.5 (#221)
hypersec-ci-bot[bot] Oct 10, 2026
ca7e4ff
fix: e2e harness restores the stack, runs via uv (#222)
catinspace-au Oct 11, 2026
c03fd4e
fix: e2e and post leave nothing behind (#223)
catinspace-au Oct 11, 2026
3d28c7c
fix(stack): pin the stack dial at 2.2.1-rc.6 (#224)
hypersec-ci-bot[bot] Oct 11, 2026
dc76c88
fix: add make proof, the unattended acceptance run for a self-updatin…
catinspace-au Oct 11, 2026
b8e0eff
fix: prove each stack version once, not on every update tick (#226)
catinspace-au Oct 11, 2026
85f3f08
fix: fast-forward the checkout on every update tick, not only on a ne…
catinspace-au Oct 12, 2026
9144479
fix(dev): run the console live with LIVE_UI=1
kazmosahebi Oct 12, 2026
b443418
fix(dev): load the live console on its own host
kazmosahebi Oct 12, 2026
bc11df0
style: split the clause commas in the live UI prose
kazmosahebi Oct 12, 2026
d7dd159
fix: hold the e2e restore to the config its files give (#231)
catinspace-au Oct 12, 2026
e66858d
fix: serialise proof and update, refuse a foreign .env owner, clone d…
catinspace-au Oct 12, 2026
c43b4cb
fix(dev): point Mac users at host next dev
kazmosahebi Oct 12, 2026
366ed7a
Merge pull request #229 from hyperi-io/fix/ui-live-mode
kazmosahebi Oct 12, 2026
8d27eed
fix: clone dfe-transform-vrl for the proof's source walk (#236)
catinspace-au Oct 13, 2026
2f7c999
fix: create the source's own transform in the source walk (#239)
catinspace-au Oct 13, 2026
defb288
fix: base slim's per-source transforms on gRPC (#240)
catinspace-au Oct 13, 2026
1de8da1
fix(make): seed sources, rules, hunts and events
kazmosahebi Oct 13, 2026
3f7df0e
fix(stack): move the stack dial to 2.2.1-rc.8 (#244)
hypersec-ci-bot[bot] Oct 13, 2026
5dab5a6
fix: record the proof against the version the stack is pinned to (#245)
catinspace-au Oct 13, 2026
d2c0cae
fix(stack): pin the stack dial at 2.2.1-rc.9 (#246)
hypersec-ci-bot[bot] Oct 13, 2026
e332ac4
fix: restore only the services the files still define (#247)
catinspace-au Oct 13, 2026
9b9ade3
fix(stack): pin the stack dial at 2.2.1-rc.10 (#251)
hypersec-ci-bot[bot] Oct 13, 2026
5c0d98f
chore: ignore local agent and tool working files (#252)
catinspace-au Oct 13, 2026
fecceba
fix: 404 /metrics and strip caller identity headers (#253)
catinspace-au Oct 14, 2026
2a6d3ba
chore(deps): Update rust:1.99-trixie Docker digest to 8101567 (#248)
renovate[bot] Oct 14, 2026
567916b
chore(deps): Update actions/create-github-app-token action to v3.3.0 …
renovate[bot] Oct 14, 2026
dee1eab
chore(deps): Update dependency ubuntu to v26 (#250)
renovate[bot] Oct 14, 2026
2c06a10
docs: say when HyperDX learns who signed in behind the auth profile (…
catinspace-au Oct 14, 2026
3884d12
fix(seed): skip what the profile cannot run
kazmosahebi Oct 15, 2026
35cbef7
Merge pull request #241 from hyperi-io/fix/make-seed-targets
kazmosahebi Oct 15, 2026
ade37c1
fix(stack): pin the stack dial at 2.2.1-rc.12 (#262)
hypersec-ci-bot[bot] Oct 15, 2026
8c03edd
fix: refuse stray origin characters, fix two docs (#263)
catinspace-au Oct 15, 2026
34bc549
fix(stack): pin the stack dial at 2.2.1-rc.13 (#264)
hypersec-ci-bot[bot] Oct 15, 2026
6507758
fix: pull the published images with no registry login (#265)
catinspace-au Oct 15, 2026
d0bc432
fix: say the component repos are public, not private until GA (#266)
catinspace-au Oct 15, 2026
6a27ef6
fix: release check dial and two post checks (#268)
catinspace-au Oct 17, 2026
dfc5d45
fix(stack): move the stack dial to 2.2.1-rc.14 (#271)
hypersec-ci-bot[bot] Oct 18, 2026
ab7f469
fix: hunt checks run for an org their events name (#270)
catinspace-au Oct 18, 2026
0cc0244
fix(security): give each app its own CH user (#269)
catinspace-au Oct 18, 2026
3c07cfb
fix: drop the removed archiver default_topic key (#267)
catinspace-au Oct 18, 2026
cf6eb1d
fix: run Keeper embedded in the bundled ClickHouse (#272)
catinspace-au Oct 18, 2026
120a85e
fix: hold the image VOLUME table to local images (#273)
catinspace-au Oct 18, 2026
95a9828
chore(deps): Update rust:1.99-trixie Docker digest to e259494 (#274)
renovate[bot] Oct 18, 2026
e28a12d
fix: pin the DFE 2.2.1-rc.15 stack (#276)
catinspace-au Oct 19, 2026
30d4410
fix: name the ClickHouse metrics receiver sql_query (#277)
catinspace-au Oct 19, 2026
9f99280
docs: say the suite repos read anonymously (#279)
catinspace-au Oct 20, 2026
1debe92
fix: pin the stack to 2.2.1-rc.17 (#281)
catinspace-au Oct 20, 2026
4ba5d7a
fix(ci): read public dfe-infra without a token so the profile check r…
catinspace-au Oct 20, 2026
85368b8
chore: keep temporary output under .tmp
kazmosahebi Oct 15, 2026
5c954f5
fix: add make certs to mint the console TLS certificate
kazmosahebi Oct 15, 2026
1aa6d30
fix(clickhouse): keep only settings that differ from the image
kazmosahebi Oct 15, 2026
38dbdf1
fix: start fetchers only per source
kazmosahebi Oct 15, 2026
3935700
chore: move the ClickHouse config under config/
kazmosahebi Oct 16, 2026
20dfa51
fix(archiver): archive by default and take the broker from env
kazmosahebi Oct 17, 2026
8700ea9
fix(hyperdx): remove the header-dev mode
kazmosahebi Oct 18, 2026
8b3b4bc
fix(loader): take ClickHouse from the compose env
kazmosahebi Oct 18, 2026
4e5eacf
fix(loader): read every source through kafka.yaml
kazmosahebi Oct 18, 2026
58ba906
fix(loader): run the outage test on kafka.yaml
kazmosahebi Oct 18, 2026
6fe06de
fix(loader): take the broker from the compose env
kazmosahebi Oct 18, 2026
a961b89
fix(archiver): drop grpc.listen, now the archiver's default
kazmosahebi Oct 18, 2026
eda061f
fix(otel): rename the deprecated receivers and tidy the collector config
kazmosahebi Oct 18, 2026
35dcf7e
fix(proxy): drop the header-dev identity strips and tidy the proxy co…
kazmosahebi Oct 19, 2026
c4629fd
fix(receiver): take the broker from compose and drop the Vector input
kazmosahebi Oct 19, 2026
3293a90
chore: move the seed data to resources/seed
kazmosahebi Oct 19, 2026
cd6197e
style: drop the comma before "and" and "or"
kazmosahebi Oct 20, 2026
e9808c0
chore: move the vector filebeat transform to resources
kazmosahebi Oct 20, 2026
eb91913
fix(clickhouse): keep only the Keeper settings that differ from the i…
kazmosahebi Oct 20, 2026
31ddf3a
chore: drop the dfe-infra profile projection
kazmosahebi Oct 20, 2026
6b62b78
fix: drop the transform configs and profiles the engine now owns
kazmosahebi Oct 20, 2026
54f4fc1
fix: collapse the service profiles to slim and single over grpc and k…
kazmosahebi Oct 21, 2026
bb23577
fix: gate per-source apps by profile flags and run local builds in th…
kazmosahebi Oct 21, 2026
1915db8
fix: drive the transform e2e tests via dfe-engine
kazmosahebi Oct 21, 2026
e04146a
fix: keep a stack e2e cannot restore running
kazmosahebi Oct 21, 2026
d242313
fix: start Kafka before the engine in e2e runs
kazmosahebi Oct 21, 2026
0ff00ef
fix: roll the archiver every 10s in kafka-outage
kazmosahebi Oct 21, 2026
1a815f2
fix: refuse e2e runs that would move the stack
kazmosahebi Oct 21, 2026
0eac2b4
fix: restore Keeper recovery docs, tidy prose
kazmosahebi Oct 22, 2026
218e7c7
fix: skip disabled sources, build per-source apps
kazmosahebi Oct 22, 2026
aed8ed5
fix: drop config keys equal to their defaults
kazmosahebi Oct 22, 2026
8e17ed9
fix: validate the proxy configs on the stack's Envoy
kazmosahebi Oct 22, 2026
1ceba3f
style: lay out the ClickHouse configs for reading
kazmosahebi Oct 22, 2026
46e26bb
fix: run no archiver on gRPC profiles
kazmosahebi Oct 23, 2026
eb39f06
fix: drop the receiver routing rules v1.15.54 refuses
kazmosahebi Oct 23, 2026
703640a
fix: give the engine the elastic contract, not a catalogue
kazmosahebi Oct 22, 2026
8b9aaa5
fix: set the receiver transport key
kazmosahebi Oct 23, 2026
e025304
fix: move the receiver configs onto scalo rules
catinspace-au Oct 22, 2026
5d4a1a3
fix: mount the engine's bearer-token directory into the receiver
catinspace-au Oct 22, 2026
cd637eb
fix: create the receiver's bearer-token directory
catinspace-au Oct 22, 2026
910763c
fix: move HyperDX's store onto DocumentDB
catinspace-au Oct 22, 2026
8e8d291
fix(compose): default the DocumentDB pin to its digest
catinspace-au Oct 22, 2026
830b55f
fix: start the source runner's make apply without the .env keys
catinspace-au Oct 23, 2026
23ada3c
fix: carry the cleanup, receiver rules and DocumentDB into the rc.19 …
catinspace-au Oct 23, 2026
1d7c1f2
fix: report cores deployed and worker-node cores on the docker stack …
catinspace-au Oct 23, 2026
0b18203
chore(deps): Update rust:1.99-trixie Docker digest to 931c2c0 (#295)
renovate[bot] Oct 25, 2026
70bcc62
chore(deps): Update actions/setup-python action to v7.1.0 (#296)
renovate[bot] Oct 25, 2026
8696350
fix: prove the vector transform on every stack version (#297)
catinspace-au Oct 25, 2026
5d0057d
fix: let the dfe-docker VM update and run proofs without sudo (#299)
catinspace-au Oct 25, 2026
07afb97
docs: check the no-sudo polkit rule by starting the unit (#301)
catinspace-au Oct 25, 2026
59e9598
fix: compare a stored rule's WHERE without identifier quoting (#302)
catinspace-au Oct 25, 2026
6332b6c
fix: timestamp dfe-docker script output in scalo's text log shape (#300)
catinspace-au Oct 25, 2026
ded9352
fix: make init.py --help print help instead of running init (#303)
catinspace-au Oct 25, 2026
e11745b
docs: say operator make stops under console TLS on a shared checkout …
catinspace-au Oct 25, 2026
4983c25
fix: fail the proof's source walk at once on a profile that runs no t…
catinspace-au Oct 26, 2026
4669b25
fix: give the e2e engine readiness poll the compose start period (#306)
catinspace-au Oct 26, 2026
e13c3fd
fix: tell dfe-engine each e2e test's bus presence, not make's (#308)
catinspace-au Oct 26, 2026
0a45ba0
docs: say why the gRPC profiles run no archiver (#307)
catinspace-au Oct 26, 2026
7ebe753
fix: cap receiver records at 8,000,000 bytes (#309)
catinspace-au Oct 26, 2026
38d4793
fix: add make demo, live vendor syslog detected by dfe-transform-elas…
catinspace-au Oct 26, 2026
a2edc27
fix: create the demo's vendor tables through the bulk catalogue endpo…
catinspace-au Oct 26, 2026
9980de2
fix: let make demo run again once its sources exist (#314)
catinspace-au Oct 26, 2026
befe669
fix: parse every demo vendor from the original syslog line, over TCP …
catinspace-au Oct 26, 2026
406c01c
fix: leave the receiver's raw capture off in make demo (#316)
catinspace-au Oct 26, 2026
a4758a7
fix: give make demo ten minutes of varied ASA and FortiGate syslog (#…
catinspace-au Oct 26, 2026
89a0829
docs: start the evaluation with make up and list uv as a prerequisite…
catinspace-au Oct 27, 2026
4b71d16
fix: strip an inline comment's padding from the .env pins make passes…
catinspace-au Oct 28, 2026
d7d6758
fix: take the tool pins and image digests to their latest (#327)
catinspace-au Oct 28, 2026
3a34c62
fix: stop publishing the broker's in-network listener to the host (#328)
catinspace-au Oct 28, 2026
9ff9f57
fix: add make targets that switch the stack in and out of the Playwri…
catinspace-au Oct 28, 2026
b28b4ad
docs: split known issues and common failures out of the troubleshooti…
catinspace-au Oct 28, 2026
bfe6eb1
fix: clear the host DNS search domain in containers (#332)
catinspace-au Oct 28, 2026
9c26eee
fix: mount a writable GeoIP volume on transform-elastic (#331)
catinspace-au Oct 28, 2026
6272153
fix: ship per-source instance logs and follow the bind host (#334)
catinspace-au Oct 28, 2026
1302b51
fix: send the gRPC receiver's dead letters to the spool only (#333)
catinspace-au Oct 28, 2026
e713624
fix: refuse a start when a published host port is already taken (#335)
catinspace-au Oct 28, 2026
a5e4ff9
fix: probe the Rust apps with bash, stop them on their contract grace…
catinspace-au Oct 31, 2026
0ef88e5
docs: manual document updates -- README hero (#339)
catinspace-au Oct 31, 2026
19450da
fix: manual remediation of LLM auto fixes -- isolate e2e runs, verify…
catinspace-au Nov 1, 2026
2b4fcfa
fix(builder): link the dev Rust builds against static C libraries (#341)
catinspace-au Nov 1, 2026
5d02ff5
fix(archiver): use DFE_ARCHIVER_ env names
catinspace-au Nov 2, 2026
fe39a60
fix(demo): start a fresh clone with make demo (#344)
catinspace-au Nov 3, 2026
9cc0a3a
fix(demo): merge chore/cleanup and package the demo (#345)
catinspace-au Nov 3, 2026
9e06976
fix(install): add a release bundle and install.sh (#346)
catinspace-au Nov 3, 2026
d77d418
fix(engine): pass settings with the DFE_ENGINE_ prefix
catinspace-au Nov 3, 2026
2b3d02d
fix(install): apply .env settings during installation
catinspace-au Nov 3, 2026
60572ee
chore(deps): Update oras-project/setup-oras action to v2.0.3 (#348)
renovate[bot] Nov 4, 2026
88730a4
fix(scripts): read the engine's page tokens and problem errors (#349)
catinspace-au Nov 4, 2026
bbf19bf
fix(proxy): give engine API calls 120 s before envoy cuts them
catinspace-au Nov 9, 2026
aa014a4
fix(stack): split component env files and simplify compose/scripts (#…
kazmosahebi Nov 10, 2026
6e5201b
fix(stack): set stack version to 2.2.2-rc.1 (#351)
hypersec-ci-bot[bot] Nov 10, 2026
6492103
fix(install): run the stack from Docker alone with an init image (#353)
catinspace-au Nov 10, 2026
944ec30
fix(release): cut dfe-docker releases again (#354)
catinspace-au Nov 10, 2026
fc34af8
docs(evaluating): name PowerShell in the quickstart
catinspace-au Nov 10, 2026
47f2a20
fix(compose): name the HyperDX service dfe-hyperdx
catinspace-au Nov 10, 2026
139024c
fix(docs): name the search console dfe-hyperdx
catinspace-au Nov 10, 2026
da05f64
chore(release): v1.0.4 [skip ci]
hypersec-ci-bot[bot] Nov 10, 2026
ee86c18
fix(demo): complete the engine's first-run setup
catinspace-au Nov 10, 2026
8f78312
fix(renovate): keep hyperi-ci references on main
catinspace-au Nov 11, 2026
2b9f2e5
fix(stack): set stack version to 2.2.2-rc.4
hypersec-ci-bot[bot] Nov 11, 2026
a291819
fix(compose): read alert destinations from the engine's secrets backend
catinspace-au Nov 11, 2026
52f72ca
fix(stack): pin stack to 2.2.2
hypersec-ci-bot[bot] Nov 12, 2026
40169b4
fix(install): publish one-line compose installs
catinspace-au Nov 12, 2026
008a825
docs(docker): drop register words
catinspace-au Nov 12, 2026
cf2e8d3
fix(compose): run one fetcher for every fetcher source
catinspace-au Nov 12, 2026
dea7977
chore(release): v1.0.5 [skip ci]
hypersec-ci-bot[bot] Nov 12, 2026
bac9406
docs(tests): say which registry calls the pin tests fake (#378)
catinspace-au Nov 12, 2026
b28bdc6
fix(otel): log HyperDX under dfe-hyperdx (#371)
kazmosahebi Nov 12, 2026
2c27935
docs(readme): introduce the repo in the README (#380)
catinspace-au Nov 13, 2026
9cf68b9
docs(readme): write the docs site link as an autolink (#381)
catinspace-au Nov 13, 2026
d80b03a
docs(versions): point at quickstart.json for the Docker floors (#382)
catinspace-au Nov 13, 2026
e729085
fix(demo): publish one compose file per demo pack (#383)
catinspace-au Nov 13, 2026
be05917
fix(demo): replay a pack's Beats sources and host their transforms (#…
catinspace-au Nov 13, 2026
0eae396
fix(demo): add the cyber demo pack's sources, events and console (#384)
catinspace-au Nov 13, 2026
f42bc19
docs(readme): link this repository's own pages on the docs site (#387)
catinspace-au Nov 13, 2026
7a45cd3
fix(demo): add the telco demo pack (#389)
catinspace-au Nov 13, 2026
863c6bc
fix(demo): add the theatre and battlespace demo pack (#386)
catinspace-au Nov 13, 2026
7add283
docs(evaluating): name the three demo packs under the demo (#388)
catinspace-au Nov 13, 2026
c208e7d
chore(release): v1.0.6 [skip ci]
hypersec-ci-bot[bot] Nov 13, 2026
c7aac00
fix(stack): pin the stack dial at 2.2.2 (#377)
hypersec-ci-bot[bot] Nov 14, 2026
b08ee19
chore(release): v1.0.7 [skip ci]
hypersec-ci-bot[bot] Nov 14, 2026
20f139a
fix(stack): refuse a prerelease stack pin on a release (#391)
catinspace-au Nov 14, 2026
8e6ece0
docs(wording): replace shape with the setting or the names (#390)
catinspace-au Nov 14, 2026
546041a
docs(docker): trim stale text and config comments (#392)
catinspace-au Nov 14, 2026
0f24d69
docs(docker): state the stack's idle memory and box size (#393)
catinspace-au Nov 14, 2026
43d1659
fix(make): block ci while a demo pack is streaming (#394)
catinspace-au Nov 14, 2026
b82f579
fix(secrets): mint 32 and 64 character alphanumeric secrets (#395)
catinspace-au Nov 14, 2026
e6563d8
fix(compose): mint the admin login and print it with one command (#396)
catinspace-au Nov 15, 2026
0339b29
fix(compose): publish syslog and flow listeners through port keys (#400)
catinspace-au Nov 15, 2026
cd82ccf
fix(compose): TLS ingest via dfe-proxy and fetcher input mount (#401)
catinspace-au Nov 15, 2026
2c5cc48
fix(compose): pass the engine the images the stack runs (#402)
catinspace-au Nov 15, 2026
73bfaa6
docs(docker): correct docs against the code (#403)
catinspace-au Nov 15, 2026
e4464ea
fix(post): check dfe-ui's metrics and correct stale comments (#404)
catinspace-au Nov 15, 2026
913927f
chore(renovate): hold the python base image on 3.14
catinspace-au Nov 16, 2026
1a66402
chore(release): v1.0.8 [skip ci]
hypersec-ci-bot[bot] Nov 16, 2026
87a9005
fix(update): remove previous stack images after a nightly update (#406)
catinspace-au Nov 16, 2026
c68f2e5
docs(docker): fix claims and redraw diagrams
catinspace-au Nov 16, 2026
980fb48
test(e2e): kill hops under load and count the loss
catinspace-au Nov 16, 2026
0ca6967
chore(merge): merge origin/fix/kill-tests into rebuilt branch
catinspace-au Nov 16, 2026
1772287
test(e2e): decode the receiver spool with zstandard
catinspace-au Nov 16, 2026
bdde776
fix(e2e): count the loader's default Kafka subscription and group
catinspace-au Nov 16, 2026
762f4a5
fix(e2e): give the transform kill tests an archiver that rolls
catinspace-au Nov 16, 2026
aa2511d
test(e2e): mark the receiver SIGTERM cases as a known issue
catinspace-au Nov 16, 2026
588f7ac
fix(e2e): stop archivers through compose and ignore known failures
catinspace-au Nov 16, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
The table of contents is too big for display.
Diff view
Diff view
  •  
  •  
  •  
65 changes: 24 additions & 41 deletions .dockerignore
Original file line number Diff line number Diff line change
@@ -1,54 +1,37 @@
# Project: dfe-docker
# File: .dockerignore
# Purpose: Build-context exclusions for any image built with this repo as context
# Purpose: Build-context allowlist for docker/dfe-docker.Dockerfile, the one build with this repo root as its context
# Language: dockerignore
#
# License: BUSL-1.1
# Copyright: (c) 2026 HYPERI PTY LIMITED
#
# Nothing in this repo currently builds with the repo root as its context:
# scripts/build_dev_images.py stages each component's source into a temp dir with
# its own STAGE_EXCLUDES, and the runtime image comes from the component repo's
# own Dockerfile. So today this file is a guard, not a fix.
# The dfe-docker image copies its whole context into the folder `init` sets up,
# so this names what that folder holds and nothing more. An allowlist, because
# a file nobody listed then stays out: .env, env/, certs/ and access-summary.md
# hold secrets and can never reach an image layer.
#
# It earns its keep the moment someone adds a root-context build -- which is
# exactly when nobody remembers to write one, and secrets (.env, env/) get
# shipped into an image layer. That is the failure this prevents.
# scripts/build_dev_images.py stages each component's source into its own temp
# context, so this file does not reach those builds.

# Secrets and local config -- the reason this file exists at all.
.env
.env.*
!.env.example
env/
certs/

# Version control and CI metadata
.git/
.github/
.gitignore
.gitleaks.toml
*

# Agent and editor artefacts
.claude/
.hyperi-ai/
.mcp.json
.vscode/
.idea/
!.env.example
!Makefile
!VERSION
!LICENSE
!compose/
!config/
!deployment.example.yaml
!docker-compose.yml
!docker/dfe-docker.requirements.txt
!env.example/
!redpanda/
!resources/
!scripts/
!service_profiles.yaml

# Local build and runtime state
.profile.mk
.tmp/
tmp/
**/target/
# Inside the kept directories: tests and bytecode never run in a deployment
scripts/tests/
**/__pycache__/
**/*.pyc

# Docs and packaging that never belong in a runtime image
docs/
*.md
LICENSE
robots.txt

# Compose definitions themselves -- an image built from this context does not
# need the stack that runs it.
docker-compose*.yml
774 changes: 192 additions & 582 deletions .env.example

Large diffs are not rendered by default.

646 changes: 507 additions & 139 deletions .github/workflows/ci.yml

Large diffs are not rendered by default.

210 changes: 210 additions & 0 deletions .github/workflows/image.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,210 @@
# Project: dfe-docker
# File: .github/workflows/image.yml
# Purpose: Build the dfe-docker image for amd64 and arm64, and on a release push it and attach quickstart.json
#
# License: BUSL-1.1 - HYPERI PTY LIMITED
# Copyright: (c) 2026 HYPERI PTY LIMITED
#
# Two images per release, built from one Dockerfile on each architecture's own
# runner. `:<version>` installs the stack deployment.example.yaml pins at the
# release tag, so it installs the same stack every time. `:latest` installs the
# newest published stack when init runs, and moves only on a release that is not
# a pre-release. A pull request that touches the image builds both on both
# architectures and pushes nothing.

name: Image

on:
release:
types: [published]
workflow_dispatch:
inputs:
tag:
description: The release tag to build and attach quickstart.json to
required: true
pull_request:
paths:
- .dockerignore
- .github/workflows/image.yml
- Makefile
- docker/dfe-docker.Dockerfile
- scripts/**

permissions:
contents: read

env:
PYTHON_VERSION: "3.14.7"
REPOSITORY: ghcr.io/hyperi-io/dfe-docker
TAG: ${{ github.event.release.tag_name || inputs.tag }}

jobs:
build:
name: Build (${{ matrix.arch }})
strategy:
fail-fast: false
matrix:
include:
- arch: amd64
runner: ubuntu-26.04
- arch: arm64
runner: ubuntu-26.04-arm
# Each architecture on its own runner, never emulated
runs-on: ${{ matrix.runner }}
timeout-minutes: 30
permissions:
contents: read
packages: write
steps:
- name: Checkout
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
with:
ref: ${{ env.TAG || github.sha }}
persist-credentials: false

- name: Set up Python
uses: actions/setup-python@6d4fd22b20465ccc0f7354476588e8df7c9ca0e0 # v7.1.0
with:
python-version: ${{ env.PYTHON_VERSION }}

- name: Set up Docker Buildx
uses: docker/setup-buildx-action@f87e5991a6d7451dcb8d9637bfbc97413f497069 # v4.4.1

- name: Name the release
id: release
run: |
version="${TAG#v}"
# Assigned on its own line so a prerelease pin fails this step, not the publish.
stack="$(python3 scripts/quickstart_json.py --print-stack)"
{
echo "version=${version:-0.0.0-pr}"
echo "stack=$stack"
echo "push=${{ github.event_name != 'pull_request' }}"
} >> "$GITHUB_OUTPUT"

- name: GHCR login
if: steps.release.outputs.push == 'true'
env:
GH_TOKEN: ${{ github.token }}
run: printf '%s' "$GH_TOKEN" | docker login ghcr.io --username "$GITHUB_ACTOR" --password-stdin

# Pushed by digest alone, so the publish job tags both architectures together
- name: Build both images
env:
PUSH: ${{ steps.release.outputs.push }}
STACK: ${{ steps.release.outputs.stack }}
VERSION: ${{ steps.release.outputs.version }}
run: |
mkdir -p "$RUNNER_TEMP/digests"
for variant in latest pinned; do
if [ "$variant" = latest ]; then
stack=latest
image="$REPOSITORY"
else
stack="$STACK"
image="$REPOSITORY:$VERSION"
fi
docker buildx build \
--platform "linux/${{ matrix.arch }}" \
--file docker/dfe-docker.Dockerfile \
--build-arg "IMAGE=$image" \
--build-arg "STACK=$stack" \
--build-arg "VERSION=$VERSION" \
--output "type=image,name=$REPOSITORY,push-by-digest=true,name-canonical=true,push=$PUSH" \
--metadata-file "$RUNNER_TEMP/$variant.json" \
.
jq -r '."containerimage.digest"' "$RUNNER_TEMP/$variant.json" > "$RUNNER_TEMP/digests/$variant-${{ matrix.arch }}"
done

- name: Hand the digests to the publish job
if: steps.release.outputs.push == 'true'
uses: actions/upload-artifact@cf430e030ddbb5b0abf93d22962f4752f3646cd9 # v7.0.2
with:
name: digests-${{ matrix.arch }}
path: ${{ runner.temp }}/digests/
if-no-files-found: error
retention-days: 1

publish:
name: Publish
needs: build
if: github.event_name != 'pull_request'
runs-on: ubuntu-26.04
timeout-minutes: 15
permissions:
contents: write
packages: write
steps:
- name: Checkout
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
with:
ref: ${{ env.TAG }}
persist-credentials: false

- name: Set up Python
uses: actions/setup-python@6d4fd22b20465ccc0f7354476588e8df7c9ca0e0 # v7.1.0
with:
python-version: ${{ env.PYTHON_VERSION }}

- name: Set up Docker Buildx
uses: docker/setup-buildx-action@f87e5991a6d7451dcb8d9637bfbc97413f497069 # v4.4.1

- name: Fetch the digests
uses: actions/download-artifact@9000827ccba6bdab643e8b6fd33ac0654aef8333 # v8.0.2
with:
pattern: digests-*
path: ${{ runner.temp }}/digests
merge-multiple: true

- name: GHCR login
env:
GH_TOKEN: ${{ github.token }}
run: printf '%s' "$GH_TOKEN" | docker login ghcr.io --username "$GITHUB_ACTOR" --password-stdin

- name: Tag both architectures
env:
GH_TOKEN: ${{ github.token }}
run: |
version="${TAG#v}"
digests="$RUNNER_TEMP/digests"
docker buildx imagetools create --tag "$REPOSITORY:$version" \
"$REPOSITORY@$(cat "$digests/pinned-amd64")" "$REPOSITORY@$(cat "$digests/pinned-arm64")"
if [ "$(gh release view "$TAG" --repo "$GITHUB_REPOSITORY" --json isPrerelease --jq .isPrerelease)" = false ]; then
docker buildx imagetools create --tag "$REPOSITORY:latest" \
"$REPOSITORY@$(cat "$digests/latest-amd64")" "$REPOSITORY@$(cat "$digests/latest-arm64")"
fi
docker buildx imagetools inspect "$REPOSITORY:$version"

# scripts/stack.py reads the stack's image pins through oras
- name: Install oras
uses: oras-project/setup-oras@5e2f2482b30b0cb3e14e67f89a80c88c659a7c63 # v2.0.3

# The compose files go into the same package under their own tags, so the image tags above stay as they are: the stack and demo, then each resources/demo-<pack> as demo-<pack>. The demo's helpers run the image just tagged, by digest
- name: Publish the compose files
env:
GH_TOKEN: ${{ github.token }}
run: |
version="${TAG#v}"
docker compose version
digest="$(docker buildx imagetools inspect "$REPOSITORY:$version" --format '{{.Manifest.Digest}}')"
moving=()
if [ "$(gh release view "$TAG" --repo "$GITHUB_REPOSITORY" --json isPrerelease --jq .isPrerelease)" = false ]; then
moving=(--moving)
fi
stack="$(python3 scripts/quickstart_json.py --print-stack)"
python3 scripts/publish_compose.py --stack "$stack" \
--image "$REPOSITORY@$digest" --out-dir "$RUNNER_TEMP/compose" \
--publish "$REPOSITORY" --release "$version" "${moving[@]}"
for dir in $(find resources -mindepth 1 -maxdepth 1 -type d -name 'demo-*' -printf '%f\n' | LC_ALL=C sort); do
python3 scripts/publish_compose.py --stack "$stack" --pack "${dir#demo-}" \
--image "$REPOSITORY@$digest" --out-dir "$RUNNER_TEMP/compose" \
--publish "$REPOSITORY" --release "$version" "${moving[@]}"
done

- name: Attach quickstart.json
env:
GH_TOKEN: ${{ github.token }}
run: |
python3 scripts/quickstart_json.py --version "${TAG#v}" --out "$RUNNER_TEMP/quickstart.json"
cat "$RUNNER_TEMP/quickstart.json"
gh release upload "$TAG" "$RUNNER_TEMP/quickstart.json" --repo "$GITHUB_REPOSITORY" --clobber
36 changes: 27 additions & 9 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,14 @@ access-summary.md
deployment.yaml
# Written by the track-latest daemon: the stack version it last applied.
.dfe-stack-applied
# Written by make proof: the stack version it last proved end to end.
.dfe-proof-passed
# Held by make proof and the daemon updater while either has the stack down.
.dfe-stack.lock
*.log
# The demo and its packs replay these sample events
!resources/demo/events/**/*.log
!resources/demo-*/events/**/*.log
dist/
dist-bin/
bin/
Expand All @@ -24,11 +31,6 @@ out/
test-results/
coverage/
.tmp/
.profile.mk
# Written by `make dev LOCAL=...`: the built components repointed at :local.
docker-compose.local.yml
# Written with the profile: one service per app instance the engine has rendered.
docker-compose.instances.yml
certs/

# Python bytecode (scripts/ generated by make dev / resolve_profile)
Expand All @@ -42,18 +44,34 @@ docs/superpowers/

# Local working files (not committed)
.claude/
hyperi-ai/
.codex/
.cursor/
.gemini/
.superpowers/
.playwright-mcp/
.mcp.json
.hyperi-ai
.worktrees/
.kube/
.secrets/
.env.*
!.env*.example
hyperi-ai/
STATE.md
CLAUDE.md
TODO.md
.ai-version
.benchmarks/
# Written at the repo root by older checkouts, now under .tmp/
/.profile.mk
/docker-compose.instances.yml
/docker-compose.local.yml

# Per-service env files: env/
env/
# Local stash of the per-service env files.
.env-dir-stashed/

# Predecessor CI clone and an inert submodule declaration, both untracked and
# left in the tree. Ignored so `git add -A` cannot pull a dead CI system (100+
# files) or a broken submodule pointer into this repo. Safe to delete outright.
# A predecessor CI clone and an inert submodule declaration, both untracked. Safe to delete.
ci/
.gitmodules
11 changes: 11 additions & 0 deletions .pytest.ini
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
# Project: dfe-docker
# File: .pytest.ini
# Purpose: Python pytest configuration file (hide temporary files)
# Language: Ini
#
# License: BUSL-1.1
# Copyright: (c) 2026 HYPERI PTY LIMITED

[pytest]
cache_dir = .tmp/pytest
addopts = -p no:benchmark
Loading
Loading