Skip to content

Fix worker cleanup and concurrent catalog publication #22

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
dnstelecom merged 2 commits into from
Sep 27, 2026
Merged

Fix worker cleanup and concurrent catalog publication #22

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
293e30a
Fix staged worker cleanup and concurrent catalog publication
dnstelecom Sep 27, 2026
7431047
Bump version
dnstelecom Sep 27, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion Cargo.lock
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 1 addition & 1 deletion Cargo.toml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@

[package]
name = "dpp"
version = "0.4.0"
version = "0.4.1"
edition = "2024"
rust-version = "1.98.1"
build = "build.rs" # Custom build script to include version information from git
Expand Down
6 changes: 6 additions & 0 deletions docs/architecture.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -83,6 +83,12 @@ and uses the remaining CPU budget for shard workers. Routed DNS packets also car
metadata into shard workers so the worker path can reuse the first L3/L4 parse instead of
repeating it for full DNS question decoding.

The staged pipeline waits for every successfully started worker before returning a startup
error. It releases the input and result channels before joining those workers, so cleanup can
finish without an aggregator. During normal teardown it joins every worker
even if an earlier worker returns an error or panics, preserving the first worker error after
any parser-stage error.

Ethernet decoding also produces one canonical VLAN context: the ordered stack of tag TPIDs and
12-bit VLAN IDs. PCP and DEI are QoS metadata and do not distinguish transactions. This context is
shared by fragment keys and matcher identities, keeping overlapping IP endpoints in different
Expand Down
6 changes: 6 additions & 0 deletions docs/synthetic-pcap-generator.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -73,6 +73,12 @@ cargo run -p dns-catalog-builder --release -- \
--top 10000
```

The catalog builder writes and syncs a separate temporary file beside the destination before
replacing the output. Concurrent builds targeting the same path cannot truncate each other's
temporary files; the last successful replacement supplies the complete catalog. Ordinary I/O
errors preserve the previous output and trigger best-effort removal of that build's temporary
file. A process crash or a filesystem cleanup error can leave a temporary file behind.

The generator loads `fitted-generator.toml` from `--profile-dir`, verifies the referenced
`catalog_data.tsv` digest, and then uses the artifact directory as the runtime source of truth.
The checked-in `server1-jul-2024` profile points its `catalog_path` at the workspace-level
Expand Down
Loading
Loading