feat(observability): Prometheus metrics, Grafana dashboards and batch-aware accounting - #113

Merged

XTLine merged 44 commits into

mainfrom

perf/observability-stack

Aug 24, 2026

Commits on Aug 21, 2026

test: add append visibility test for Latest cursor
XTLine
committed
e366988
perf(storage): publish LAC via ArcSwap and drop ledger content locks
XTLine
committed
793466e
Remove unneeded mut binding
XTLine
committed
29614f1
perf(storage): cap RocksDB memory usage
XTLine
committed
bd8fe06
perf storage leak between scenarios
XTLine
committed
15b4303
test: increase stress test offered load
XTLine
committed
dba75f2
test: Add perf log parsing and E2E diagnostics tests
XTLine
committed
21f9e3a
Refactor perf parsing and error reporting
XTLine
committed
1d63ef7
feat: add external broker backend and local cgroup limits
XTLine
committed
9867496
Monitor external broker via systemd unit and cgroup
XTLine
committed
15fca7f
Update perf stress scripts for external broker runs
XTLine
committed
1834646
Add TODO comment in read_position
XTLine
committed
7137100
fix(broker): bound shared dispatcher memory with unacked gate and merge-trigger
XTLine
committed
55906cd
perf(broker): switch to jemalloc to avoid glibc arena memory retention
XTLine
committed
1534a36
style: cargo fmt
XTLine
committed
bd3bf6d
feat(proto): extract pulsar binary protocol into pulsar-lite-proto crate
XTLine
committed
180470e
refactor(broker): remove duplicated protocol/storage modules from main crate
XTLine
committed
2c73988
fix(broker): resolve orphan-rule violation via ConsumerMessage newtype; move seek helpers into storage core
XTLine
committed
407b697
refactor(broker): migrate imports to extracted pulsar-lite-proto and storage crates
XTLine
committed
11c4f26
style: cargo fmt workspace formatting
XTLine
committed
e5213af
perf(broker): decouple dispatch from send-receipt path
XTLine
committed
5da74d6
perf(broker): remove locks from persistent enqueue hot path
XTLine
committed
87bdd4c
perf(storage): coalesce individual acks into ranges
XTLine
committed
6e5af16
debug(broker): add TEMP DIAG file-based probes for ack pipeline
XTLine
committed
b3f244b
fix(broker): pass persistent_appender in test connection builder
XTLine
committed
aa8b6d8
Fix permit accounting and backpressure in broker dispatch
XTLine
committed
c5c16e1
Add pending publish byte limits and connection write-state config
XTLine
committed
cc4b065
chore(broker): remove TEMP DIAG instrumentation probes
XTLine
committed
970e76b
style: cargo fmt and clippy 1.98 fixes (while-let loop in cursor)
XTLine
committed
6acc438
fix(broker): restore persistent_appender arg and drop redundant u32 casts for clippy 1.98
XTLine
committed
b4dc1c2
fix(broker): apply flow permits synchronously under the subscription lock
XTLine
committed
3acae83
fix(broker): enable TCP_NODELAY on accepted sockets
XTLine
committed
6a9a8eb
perf(broker): ordered batched fan-out worker for non-persistent publish
XTLine
committed
8daa9e9
chore(broker): demote per-Flow log to debug
XTLine
committed
21238db
perf(broker): batch persistent send receipts into single flushes
XTLine
committed
3e84dc5
feat(metrics): add pulsar-lite-metrics crate with storage-side families
XTLine
committed
7f621af
feat(broker): wire Prometheus metrics across broker layers
XTLine
committed
c4d1337
feat(observability): add Grafana dashboards and compose stack
XTLine
committed
8e61aa3
docs: add monitoring section to README
XTLine
committed
bf11655
fix(perf): give each perf broker a dedicated scrape-able metrics port
XTLine
committed
1b9980f
feat(broker): batch-aware message accounting across dispatch and commit paths
XTLine
committed
e542f06
fix(stats): keep previous gauge value when storage lock is busy
XTLine
committed
98189d6
chore(grafana): expose Grafana on host port 7070 to avoid 3000 conflicts
XTLine
committed
086120b
style: cargo fmt and clippy 1.98 fixes (checked_div in metrics)
XTLine
committed
c0003ad