Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 5 additions & 1 deletion .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -111,6 +111,10 @@ jobs:
- mysql,tracing
- postgres,streaming
- postgres,webhooks
- postgres,full
- mysql,full
- full
- test-queue
steps:
- uses: actions/checkout@v5
- name: Install system packages
Expand Down Expand Up @@ -141,7 +145,7 @@ jobs:
run: cargo test --workspace --all-features --lib --bins --no-run
- name: Run unit tests (no databases)
run: cargo test --workspace --all-features --lib --bins --no-fail-fast
- name: Doctests with default features
- name: Doctests with the default (empty) feature set
run: cargo test -p hammerwork --doc

# Database jobs: every test file under tests/ contains both a PostgreSQL and a MySQL module,
Expand Down
14 changes: 14 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,20 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0

## [Unreleased]

### Breaking changes

- **No features are enabled by default** (#92). `default` used to be `["metrics", "alerting", "webhooks"]`, which pulled `warp`, `reqwest`, `lettre` and `prometheus` into every build. It is now empty. Enable what you use, or the new `full` feature (`metrics`, `alerting`, `webhooks`, `streaming`, `tracing`, `encryption`; no database backend, KMS key source, event stream sink or `test-queue`). Code that uses `hammerwork::metrics`, `hammerwork::alerting` or `hammerwork::webhooks` must now enable those features. `cargo-hammerwork` and `hammerwork-web` enable the features they need themselves.
- **`azure-kv` is renamed `azure-kms`**, to match `aws-kms`, `gcp-kms` and `vault-kms` (#92). There is no alias. The same rename applies to the `cargo-hammerwork` feature.
- **`test` is renamed `test-queue`**, after the `TestQueue` it enables (#92). There is no alias.

### Fixed

- Building with `encryption` but no database backend (and so with `full`) no longer fails clippy with dead-code and unused-import errors.

### Changed

- CI checks the feature combinations `full`, `postgres,full`, `mysql,full` and `test-queue`.

## [2.5.0] - 2026-10-11

A bug-fix release from a third bug hunt (#111 to #114, #118). It is a minor version
Expand Down
12 changes: 8 additions & 4 deletions Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -139,7 +139,11 @@ lettre = { version = "0.11", default-features = false, features = ["builder", "s
google-cloud-gax = { version = "1.4", package = "gcloud-gax", optional = true }

[features]
default = ["metrics", "alerting", "webhooks"]
# No features by default: just the queue, worker and job types. Pick a database backend
# (`postgres` and/or `mysql`) and opt in to subsystems; `full` turns on all of the pure-Rust
# ones (not the database backends, the KMS key sources, the event-stream sinks or `test-queue`).
default = []
full = ["metrics", "alerting", "webhooks", "streaming", "tracing", "encryption"]
postgres = ["sqlx/postgres"]
mysql = ["sqlx/mysql"]
metrics = ["prometheus", "warp"]
Expand All @@ -149,13 +153,13 @@ encryption = ["aes-gcm", "chacha20poly1305", "argon2", "hmac", "sha2", "hex", "z
aws-kms = ["aws-sdk-kms", "aws-config"]
gcp-kms = ["google-cloud-kms", "google-cloud-auth", "rustls"]
vault-kms = ["vaultrs", "url"]
azure-kv = ["azure_security_keyvault_keys", "azure_identity", "azure_core"]
azure-kms = ["azure_security_keyvault_keys", "azure_identity", "azure_core"]
kafka = ["rdkafka"]
google-pubsub = ["google-cloud-pubsub", "google-cloud-auth", "google-cloud-googleapis", "google-cloud-gax", "rustls"]
kinesis = ["aws-sdk-kinesis", "aws-config"]
tracing = ["opentelemetry", "opentelemetry_sdk", "opentelemetry-otlp", "tracing-opentelemetry"]
streaming = ["rmp-serde", "apache-avro", "prost"]
test = []
test-queue = []

[dev-dependencies]
tokio-test = { workspace = true }
Expand Down Expand Up @@ -207,7 +211,7 @@ required-features = ["postgres"]

[[example]]
name = "test_queue_example"
required-features = ["test"]
required-features = ["test-queue"]

[[example]]
name = "encryption_example"
Expand Down
50 changes: 30 additions & 20 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -32,36 +32,46 @@ A high-performance, database-driven job queue for Rust with comprehensive featur

### Core Library

No features are enabled by default: Hammerwork 3.0 gives you the queue, worker, job, retry, cron, batch, workflow and priority APIs, without the HTTP, SMTP, Prometheus, OpenTelemetry or encryption dependencies. You always pick a database backend, and opt in to the subsystems you use.

```toml
[dependencies]
# Default features include metrics and alerting
# A queue on PostgreSQL (or MySQL): the lightest build
hammerwork = { version = "2.5", features = ["postgres"] }
# or
hammerwork = { version = "2.5", features = ["mysql"] }

# With encryption for PII protection
hammerwork = { version = "2.5", features = ["postgres", "encryption"] }

# With AWS KMS integration for enterprise key management
hammerwork = { version = "2.5", features = ["postgres", "encryption", "aws-kms"] }

# With Google Cloud KMS integration for enterprise key management
hammerwork = { version = "2.5", features = ["postgres", "encryption", "gcp-kms"] }

# With HashiCorp Vault KMS integration for enterprise key management
hammerwork = { version = "2.5", features = ["postgres", "encryption", "vault-kms"] }
# Everything that is not a backend, a key source or an event sink:
# metrics, alerting, webhooks, streaming, tracing and encryption
hammerwork = { version = "2.5", features = ["postgres", "full"] }

# With distributed tracing
hammerwork = { version = "2.5", features = ["postgres", "tracing"] }
# Or pick subsystems one by one
hammerwork = { version = "2.5", features = ["postgres", "metrics", "alerting", "webhooks"] }

# Full feature set
hammerwork = { version = "2.5", features = ["postgres", "encryption", "aws-kms", "gcp-kms", "vault-kms", "tracing"] }
# With encryption for PII protection, and a KMS key source
hammerwork = { version = "2.5", features = ["postgres", "encryption", "aws-kms"] } # or "gcp-kms", "vault-kms", "azure-kms"

# Minimal installation
hammerwork = { version = "2.5", features = ["postgres"], default-features = false }
# In-memory TestQueue for your tests
[dev-dependencies]
hammerwork = { version = "2.5", features = ["test-queue"] }
```

**Feature Flags**: `postgres`, `mysql`, `metrics` (default), `alerting` (default), `encryption` (optional), `aws-kms` (optional), `gcp-kms` (optional), `vault-kms` (optional), `tracing` (optional), `test` (for TestQueue)
**Feature flags**

| Feature | Enables |
|---|---|
| `postgres`, `mysql` | database backends (pick at least one to use a real queue) |
| `metrics` | Prometheus metrics and the metrics HTTP endpoint |
| `alerting` | alert delivery by webhook and email |
| `webhooks` | signed webhook delivery of job events |
| `streaming` | event streaming (MessagePack, Avro, Protobuf) |
| `tracing` | OpenTelemetry tracing |
| `encryption` | payload encryption and key management |
| `aws-kms`, `gcp-kms`, `vault-kms`, `azure-kms` | KMS key sources for `encryption` |
| `kafka`, `google-pubsub`, `kinesis` | event stream sinks for `streaming` |
| `test-queue` | the in-memory `TestQueue` |
| `full` | `metrics`, `alerting`, `webhooks`, `streaming`, `tracing` and `encryption` |

`full` does not include a database backend, a KMS key source, an event stream sink (these pull in cloud SDKs or native libraries) or `test-queue`. Upgrading from 2.x: see [Upgrading to 3.0](docs/upgrading-to-3.0.md).

### Web Dashboard (Optional)

Expand Down
4 changes: 2 additions & 2 deletions cargo-hammerwork/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@ name = "cargo-hammerwork"
path = "src/main.rs"

[dependencies]
hammerwork = { workspace = true, features = ["postgres", "mysql", "encryption"] }
hammerwork = { workspace = true, features = ["postgres", "mysql", "encryption", "webhooks"] }
clap = { workspace = true, features = ["env"] }
tokio = { workspace = true }
anyhow = { workspace = true }
Expand All @@ -42,7 +42,7 @@ base64 = { workspace = true }
aws-kms = ["hammerwork/aws-kms"]
gcp-kms = ["hammerwork/gcp-kms"]
vault-kms = ["hammerwork/vault-kms"]
azure-kv = ["hammerwork/azure-kv"]
azure-kms = ["hammerwork/azure-kms"]

[dev-dependencies]
tempfile = "3.8"
Expand Down
2 changes: 1 addition & 1 deletion cargo-hammerwork/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -154,7 +154,7 @@ or `--pii-field <field>` (repeatable) encrypt a job on any queue. The CLI never
plaintext by mistake: with encryption enabled but the key unavailable, these commands fail,
and without any settings they still refuse to write a plaintext job to a queue that already
holds encrypted jobs. KMS key sources need the matching cargo feature of `cargo-hammerwork`
(`aws-kms`, `gcp-kms`, `vault-kms`, `azure-kv`).
(`aws-kms`, `gcp-kms`, `vault-kms`, `azure-kms`).

### Job Management Commands

Expand Down
6 changes: 3 additions & 3 deletions docs/encryption.md
Original file line number Diff line number Diff line change
Expand Up @@ -258,7 +258,7 @@ let config = EncryptionConfig::new(EncryptionAlgorithm::AES256GCM)
# }
```

Credentials come from the environment (`AZURE_TENANT_ID` / `AZURE_CLIENT_ID` / `AZURE_CLIENT_SECRET`, workload identity, managed identity, or the Azure CLI). Requires the `azure-kv` feature. The key material must be exactly the algorithm's key size (32 bytes, e.g. a 256-bit `oct` key); shorter or longer material is rejected with `EncryptionError::InvalidConfiguration` instead of being padded or truncated, for engine keys and `KeyManager` master keys alike.
Credentials come from the environment (`AZURE_TENANT_ID` / `AZURE_CLIENT_ID` / `AZURE_CLIENT_SECRET`, workload identity, managed identity, or the Azure CLI). Requires the `azure-kms` feature. The key material must be exactly the algorithm's key size (32 bytes, e.g. a 256-bit `oct` key); shorter or longer material is rejected with `EncryptionError::InvalidConfiguration` instead of being padded or truncated, for engine keys and `KeyManager` master keys alike.

#### Key loading fails closed

Expand All @@ -267,7 +267,7 @@ Loading a key from an external source never falls back to another key. `Encrypti
- the KMS or vault is unreachable, or rejects the credentials
- the key, secret or `key` field does not exist
- the source is malformed (for example a Vault path without a mount, or a GCP resource that is not `projects/.../locations/...`)
- the cargo feature for the source (`aws-kms`, `gcp-kms`, `vault-kms`, `azure-kv`) is not enabled
- the cargo feature for the source (`aws-kms`, `gcp-kms`, `vault-kms`, `azure-kms`) is not enabled

Previously, these cases logged an error and used a key derived from the source string (key ID, region, Vault path, vault URL). Anyone who knew the configuration could derive that key. For development without a KMS, use `KeySource::Static` with a base64-encoded key, or `KeySource::Generated`.

Expand Down Expand Up @@ -413,7 +413,7 @@ let pool = WorkerPool::from_hammerwork_config(worker, &config)?;

### Testing with `TestQueue`

The in-memory `TestQueue` (feature `test`) applies the same semantics when given an engine, so unit tests exercise the real redaction and decryption paths:
The in-memory `TestQueue` (feature `test-queue`) applies the same semantics when given an engine, so unit tests exercise the real redaction and decryption paths:

```rust,no_run
# #[allow(unused_imports)] use hammerwork::{*, encryption::*, queue::DatabaseQueue, worker::JobHandler};
Expand Down
2 changes: 1 addition & 1 deletion docs/job-types.md
Original file line number Diff line number Diff line change
Expand Up @@ -280,7 +280,7 @@ In-memory results are faster to store than database results, but:
`ResultConfig::with_max_size` is recorded with the job but not enforced, for database
and in-memory results alike.

The `TestQueue` (feature `test`) keeps all results in memory regardless of
The `TestQueue` (feature `test-queue`) keeps all results in memory regardless of
`ResultStorage`, expiring them by its mock clock.

## Error Handling
Expand Down
21 changes: 12 additions & 9 deletions docs/monitoring.md
Original file line number Diff line number Diff line change
Expand Up @@ -219,17 +219,20 @@ metrics_collector.observe_custom_histogram("custom_histogram_1", "queue_name", 1
# }
```

## Disabling Monitoring
## Enabling Monitoring

If you want to disable monitoring features:
Monitoring is opt-in: no feature is enabled by default.

```toml
# Disable all monitoring
hammerwork = { version = "0.6", features = ["postgres"], default-features = false }
# The queue only, no monitoring
hammerwork = { version = "2.5", features = ["postgres"] }

# Enable only metrics
hammerwork = { version = "0.6", features = ["postgres", "metrics"], default-features = false }
# Only metrics
hammerwork = { version = "2.5", features = ["postgres", "metrics"] }

# Enable only alerting
hammerwork = { version = "0.6", features = ["postgres", "alerting"], default-features = false }
```
# Only alerting
hammerwork = { version = "2.5", features = ["postgres", "alerting"] }

# Metrics, alerting and the other optional subsystems
hammerwork = { version = "2.5", features = ["postgres", "full"] }
```
6 changes: 3 additions & 3 deletions docs/quick-start.md
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,7 @@ Add Hammerwork to your `Cargo.toml`:

```toml
[dependencies]
# PostgreSQL support with default features (metrics, alerting)
# PostgreSQL support (no features are on by default; the queue only)
hammerwork = { version = "2.5", features = ["postgres"] }

# MySQL support
Expand All @@ -29,8 +29,8 @@ hammerwork = { version = "2.5", features = ["mysql"] }
# With distributed tracing
hammerwork = { version = "2.5", features = ["postgres", "tracing"] }

# Minimal installation without default features
hammerwork = { version = "2.5", features = ["postgres"], default-features = false }
# Metrics, alerting, webhooks, streaming, tracing and encryption together
hammerwork = { version = "2.5", features = ["postgres", "full"] }

# Additional dependencies for examples
serde_json = "1.0"
Expand Down
2 changes: 1 addition & 1 deletion docs/testing.md
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,7 @@ Add TestQueue to your test dependencies:

```toml
[dependencies]
hammerwork = { version = "2.5", features = ["test"] }
hammerwork = { version = "2.5", features = ["test-queue"] }

[dev-dependencies]
tokio-test = "0.4"
Expand Down
48 changes: 48 additions & 0 deletions docs/upgrading-to-3.0.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,48 @@
# Upgrading from 2.x to 3.0

This page lists what to change in your project when you move from Hammerwork 2.x to 3.0.

## Feature flags

Three changes to the Cargo features of `hammerwork` (#92):

1. **No default features.** In 2.x, `default = ["metrics", "alerting", "webhooks"]`. In 3.0, `default = []`, so a plain `hammerwork` dependency builds only the queue, worker and job APIs, without `warp`, `reqwest`, `lettre` or `prometheus`.
2. **`azure-kv` is now `azure-kms`**, like `aws-kms`, `gcp-kms` and `vault-kms`. There is no alias.
3. **`test` is now `test-queue`.** There is no alias.

New: a `full` feature that enables `metrics`, `alerting`, `webhooks`, `streaming`, `tracing` and `encryption`. It does not include a database backend (`postgres`, `mysql`), a KMS key source, an event stream sink (`kafka`, `google-pubsub`, `kinesis`) or `test-queue`.

### What to change

If your 2.x `Cargo.toml` relied on the defaults, name the features you use:

```toml
# 2.x (implicitly metrics + alerting + webhooks)
hammerwork = { version = "2", features = ["postgres"] }

# 3.0, same behaviour
hammerwork = { version = "3", features = ["postgres", "metrics", "alerting", "webhooks"] }

# 3.0, or everything that is not a backend, KMS source or event sink
hammerwork = { version = "3", features = ["postgres", "full"] }
```

If you used `default-features = false`, you can drop it.

Rename the other features:

```toml
# 2.x
hammerwork = { version = "2", features = ["postgres", "encryption", "azure-kv"] }
[dev-dependencies]
hammerwork = { version = "2", features = ["test"] }

# 3.0
hammerwork = { version = "3", features = ["postgres", "encryption", "azure-kms"] }
[dev-dependencies]
hammerwork = { version = "3", features = ["test-queue"] }
```

A compile error such as `unresolved import hammerwork::metrics`, `hammerwork::alerting` or `hammerwork::webhooks`, or a missing `TestQueue`, means a feature is not enabled. `cargo-hammerwork` and `hammerwork-web` already enable the features they need.

If you gate your own code with `cfg(feature = "...")` on features that you forward to hammerwork, rename them in the same way.
2 changes: 1 addition & 1 deletion docs/webhooks.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# Webhooks

`WebhookManager` (module `hammerwork::webhooks`, feature `webhooks`, on by default)
`WebhookManager` (module `hammerwork::webhooks`, feature `webhooks`, included in `full`)
delivers job lifecycle events to HTTP endpoints, with retries, authentication and
HMAC signatures. (Alert notifications are a separate system; see
[Monitoring & Alerting](monitoring.md).)
Expand Down
2 changes: 1 addition & 1 deletion hammerwork-web/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -68,7 +68,7 @@ auth = ["bcrypt"]

[dev-dependencies]
# `TestQueue` (in-memory queue) for handler tests
hammerwork = { workspace = true, features = ["test"] }
hammerwork = { workspace = true, features = ["test-queue"] }
tempfile = { workspace = true }
tokio-test = { workspace = true }
warp = { workspace = true, features = ["test"] }
2 changes: 1 addition & 1 deletion src/archive.rs
Original file line number Diff line number Diff line change
Expand Up @@ -1528,7 +1528,7 @@ mod tests {
/// A scheduled pass whose policy ages are too large to subtract from now (say
/// `Duration::MAX` for "never purge") archives and purges nothing instead of
/// panicking.
#[cfg(all(feature = "test", feature = "postgres"))]
#[cfg(all(feature = "test-queue", feature = "postgres"))]
#[tokio::test]
async fn scheduled_pass_with_huge_ages_does_not_panic() {
let queue = crate::queue::test::TestQueue::new();
Expand Down
2 changes: 1 addition & 1 deletion src/config.rs
Original file line number Diff line number Diff line change
Expand Up @@ -1095,7 +1095,7 @@ impl std::str::FromStr for PayloadEncryptionAlgorithm {
/// - `aws://...`, `gcp://...`: a data key generated by AWS or GCP KMS and stored
/// KMS-encrypted in the database (`aws-kms` / `gcp-kms` features)
/// - `vault://...`: a HashiCorp Vault secret (`vault-kms` feature)
/// - `azure://...`: an Azure Key Vault secret (`azure-kv` feature)
/// - `azure://...`: an Azure Key Vault secret (`azure-kms` feature)
///
/// Static keys cannot be configured: a key written in a configuration file would be
/// printed and saved with it. Anything else is rejected when the configuration is
Expand Down
6 changes: 3 additions & 3 deletions src/encryption.rs
Original file line number Diff line number Diff line change
Expand Up @@ -54,7 +54,7 @@
//! # }
//! ```

#[cfg(feature = "azure-kv")]
#[cfg(feature = "azure-kms")]
mod azure;
pub mod engine;
mod envelope;
Expand Down Expand Up @@ -991,7 +991,7 @@ impl EncryptionStats {
///
/// # Arguments
///
/// * `service_prefix` - Service-specific prefix (e.g., "aws-kms-data-key", "azure-kv-master-key")
/// * `service_prefix` - Service-specific prefix (e.g., "aws-kms-data-key", "azure-kms-master-key")
/// * `inputs` - Variable number of string inputs that will be hashed together
///
/// # Returns
Expand All @@ -1010,7 +1010,7 @@ impl EncryptionStats {
/// assert_eq!(key.len(), 32); // SHA-256 produces 32 bytes
///
/// // Different inputs give a different key
/// let key = generate_deterministic_key("azure-kv-master-key", &["vault-url", "key-name"]);
/// let key = generate_deterministic_key("azure-kms-master-key", &["vault-url", "key-name"]);
/// assert_eq!(key.len(), 32);
/// # }
/// ```
Expand Down
Loading
Loading