Merged
Conversation
Fifteen anonymised single-line JSON events under tests/fixtures/vl_events/ cover the shapes encountered in practice: nginx flat-dotted, cisco regex- extractable, k8s with label keys, Windows PascalCase, journald _SYSTEMD_*, OpenTelemetry (randomised IDs, not the W3C canonical examples), raw _msg only, JSON-embedded _msg, edge cases (unicode CJK + emoji, empty fields, literal dotted keys), audit, docker, syslog daemon. A YAML manifest documents each fixture (description, source_system, tags, prevents_regression_for). The helper module tests/common/vl_events.rs exposes load_fixture, load_fixtures_by_tag, load_fixtures_by_source_system, and all_fixtures, with the manifest cached lazily via OnceLock and indexed as a BTreeMap for determinism across parallel test runs. Two integration test files ship the first consumers. The consistency test checks index <-> filesystem coherence, one-line JSON shape, required VL fields, and tag/source enumeration. The parameterised template test exercises a byte-exact render of _msg/_time against the whole corpus, the #25 dotted-key regression on the dotted_keys tag, the empty-field contract on edge_empty_fields, the lenient-undefined contract on raw sources, and unicode round-tripping on the unicode fixture. This delivers deferred item D-25-2 and lays groundwork for v2.0.0 multi- source testing.
chore(tests): add VictoriaLogs event fixtures corpus (D-25-2)
`victorialogs:` becomes a `BTreeMap<String, VlSourceConfig>` indexed by
source name. Rules gain an optional `vl_sources: [name, ...]` field;
absent means fan out to every configured source. The engine spawns one
task per (rule, source) pair with per-source cancellation and reconnect
isolation, so a single unhealthy source does not stop alerts on the
others.
Synthetic `vl_source` is injected into every render context where
`rule_name` already flowed: layer 1 templates (title, body,
email_body_html), throttle.key, and notifier-level layer 2 contexts.
The default throttle key becomes `{rule}-{source}:global` so multi-source
deployments get isolated buckets per source out of the box.
Source names are restricted to `^[a-zA-Z0-9_]+$` to keep the default
throttle key separator (`-`) unambiguous. Rules reject duplicate
`vl_sources` entries at load. The legacy single-URL config shape is
rejected with an actionable migration error.
`AlertPayload.vl_source` propagates end-to-end. The Mattermost footer
gains a source segment and the default webhook payload exposes
`vl_source` as a top-level JSON field. See CHANGELOG v2.0.0 Breaking
section for the full migration.
Tests consume the fixtures corpus (PR #35). The new
`tests/multi_source_integration.rs` exercises the full routing matrix
across two wiremock VL backends, including a negative-evidence assertion
that a pinned rule does not leak to the unpinned source.
Observability and guardrails (metrics `vl_source` label,
`valerter_vl_source_up` gauge, `max_streams` cap, reconnect jitter) ship
as a separate spec part 2 before the v2.0.0 tag.
Cargo.toml not bumped here; version bump lives in the release PR.
feat!: multi-source VictoriaLogs (v2.0.0 part 1, #34)
Adds the `vl_source` Prometheus label to every per-rule counter, replaces
the legacy `valerter_victorialogs_up{rule_name}` gauge with a per-source
`valerter_vl_source_up{vl_source}`, introduces a `defaults.max_streams`
cap with load-time enforcement, applies +/-10% uniform jitter on reconnect
backoff per (rule, source) task to break thundering-herd alignment, and
ships a `/metrics` snapshot integration test that catches accidental
relabel or rename in future PRs.
Multi-source operators can now attribute every alert and error rate to a
specific source. The new gauge makes partial-source outages visible
without requiring per-rule reachability inference. The cap prevents an
accidental (rule x source) fan-out from DoS'ing a backend at startup.
Post-review hardening folded in: dated v1.2.1 in CHANGELOG, added
PromQL migration snippet for dashboards, extended snapshot coverage
to include lines_discarded / query_duration / last_query_timestamp /
per-notifier sentinel counters, made the jitter floor test exercise the
actual clamp branch, demoted backoff_delay_default to pub(crate),
extended ReconnectCallback::on_reconnect to receive vl_source, and
rejected max_streams=0 at load with a clear error.
Cargo.toml not bumped here; version bump lives in the release PR.
…dy_html Documents a pre-existing concern made wider by the v1.2.0 #26 fix: the example config now actively pipes `{{ _msg }}` into `body`, and operators who mirror this pattern in `email_body_html` may inadvertently render unescaped HTML/script content from untrusted log fields. Hardening of the email path is a follow-up; the advisory makes the limit visible at the v2.0.0 release boundary.
feat!: multi-source observability and guardrails (v2.0.0 part 2, #34)
…ATION + multi-source sample Three v1.x example configs (README Quick Start, docs/getting-started.md minimal, examples/cisco-switches/config.yaml) were rejected by --validate on v2.0.0; they are now migrated to the map shape with a `default:` source. Adds a top-level MIGRATION.md with a 5-section guide for v1.x upgraders: pre-upgrade checklist, config migration with before/after snippets, Prometheus metric migration with PromQL examples, notifier output format changes (Mattermost footer + default webhook payload), and a rollback path. Linked from README and CHANGELOG. Adds examples/multi-source/ with a complete reference config and a README explaining the routing matrix (one rule pinned to prod, one pinned to staging, one fan-out across both). Closes the explicit ask from the issue #34 reporter for a sample multi-source config. README Features list hoists "One Valerter for every VictoriaLogs you run" to position #1, reworded outcome-led. A blockquote callout right after Quick Start points v1.x upgraders at MIGRATION.md. docs/performance.md gains an accuracy banner since the throughput numbers were measured against v1.x (single task per rule); the multi-source N x M task model and max_streams cap are mentioned and will be re-measured separately. Two new integration tests in tests/integration_validate.rs lock the shipped samples against future drift: shipped_config_example_validates and shipped_examples_pass_validate (iterates examples/*/config.yaml). The 2.0.0 entry in CHANGELOG gains one Added line pointing at the new sample and MIGRATION.md.
chore(docs): v2.0.0 doc consolidation, fix stale examples + ship MIGRATION + multi-source sample
Codecov Report❌ Patch coverage is
📢 Thoughts on this report? Let us know! |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Promotes
release/2.0.0tomain. Major version bundling the v2.0.0-rc1 work plus the final doc consolidation, validated end-to-end by the issue #34 reporter.Bundled
victorialogs:becomes a map of named sources. Rules gain optionalvl_sources: [name, ...]. Engine spawns one task per(rule, source)pair with per-source cancellation and reconnect isolation.{{ vl_source }}injected into every template context where{{ rule_name }}flowed. Default throttle key changes to{rule}-{source}:global. Legacy single-URL shape rejected at load with a migration-oriented error.vl_sourcelabel added to every per-rule Prometheus counter.valerter_victorialogs_up{rule_name}replaced byvalerter_vl_source_up{vl_source}per-source gauge.defaults.max_streams: 50cap with load-time enforcement. ±10% uniform jitter on reconnect backoff per(rule, source)task.tests/metrics_snapshot.rsintegration test locks metric names + label keys against drift.MIGRATION.mdat repo root with v1→v2 upgrade guide.examples/multi-source/reference config. README Features reorder. Accuracy banner ondocs/performance.md. Two new integration tests lock shipped samples against drift.Validation
/metricsoutput schema;max_streamsrejection verified live.config/config.example.yaml,examples/cisco-switches/config.yaml,examples/multi-source/config.yaml) validate. README Quick Start +docs/getting-started.mdminimal config both validate on copy-paste.Breaking changes
See
CHANGELOG.mdsection## [2.0.0]for the full text. Short list:victorialogs:shape change (mandatory migration via map of named sources).{rule}:global→{rule}-{source}:global.^[a-zA-Z0-9_]+$.vl_sourcelabel.valerter_victorialogs_up{rule_name}removed, replaced byvalerter_vl_source_up{vl_source}.defaults.max_streams: 50default cap added.MIGRATION.mdat the repo root has the full upgrade checklist with PromQL examples.Release flow
After this merge,
v2.0.0gets tagged on the merge commit and the release build produces the final binaries and debs.