Skip to content

Dashboard DISTINCT counts, equity/freshness panels, structured precursor events (fixes #224)#237

Merged
wjllance merged 1 commit into
mainfrom
fix-224-dashboard-distinct-panels
Jul 13, 2026
Merged

Dashboard DISTINCT counts, equity/freshness panels, structured precursor events (fixes #224)#237
wjllance merged 1 commit into
mainfrom
fix-224-dashboard-distinct-panels

Conversation

@wjllance

Copy link
Copy Markdown
Owner

Summary

Closes the four dashboard + log-pipeline gaps in issue #224.

  1. DISTINCT counts — the Fills, Alerts, Cleanups, Cancel Reasons and Inventory Exits panels now use count(DISTINCT event_id) instead of count(*), matching the guidance in docs/15-openobserve.md. Daily duplicate uploads on network jitter (at-least-once retry) no longer inflate key metrics. The Cleanups metric also filters event='complete' so it is not skewed by the new retry-incomplete precursor rows.
  2. New panels
    • Equity / uPnL / Available trend (Overview), consuming cycle_summary.account (live runs only; account is null in paper mode).
    • Data Freshness (Overview) — max(_timestamp) + event count for the selected run.
    • Rejections & Error Signals (Runs & Events) — order/startup rejections, warning/critical risk notifications, and reconciliation/cleanup precursor failures.
    • Stream Health / Reconnects (Runs & Events) — order-response reconnect lifecycle by phase.
  3. Structured precursor events — the two named precursor eprintln! signals now emit structured stdout JSON events in JSON mode (so the existing stdout ingest uploads them), while keeping the human-readable stderr line for non-JSON output, matching the existing emit_* pattern:
    • cancel-retry failures — recovery.rs, action=maker_cleanup, event=retry_incomplete, severity=warning.
    • reconciliation-snapshot failures — runtime.rs, action=position_reconciliation, event=snapshot_failed, severity=warning.
  4. Ingest edge cases — checkpoints now record file identity (inode + size) and reset to zero on rotation/truncation; a corrupt state file auto-resets instead of wedging --follow at startup; and the checkpoint map is bounded via LRU eviction so openobserve-uploaded.json no longer grows unbounded. event_id idempotency remains the dedup safety net. Legacy int-valued state entries are still honored.

The STANDX_ENABLE_LIVE_MAKER live lock is untouched.

Test plan

  • cargo fmt -- --check, cargo clippy --workspace --all-targets -- -D warnings, cargo test -p standx-cli — all green.
  • python3 -m py_compile scripts/openobserve_dashboard.py scripts/openobserve_ingest.py scripts/test_openobserve_ingest.py.
  • python3 -m unittest test_openobserve_ingest — 7 tests pass, including new coverage for truncation reset, corrupt-state auto-reset, legacy int state, and LRU bounding.
  • Offline build_dashboard() smoke check: JSON-serializable, unique panel ids/layout indices per tab (Overview 13 panels, Runs & Events 7).

Fixes #224

🤖 Generated with Claude Code

… precursor events (#224)

Addresses the four dashboard + log-pipeline gaps from issue #224.

1. Count panels now use count(DISTINCT event_id) instead of count(*)
   (Fills, Alerts, Cleanups, Cancel Reasons, Inventory Exits), so
   at-least-once duplicate uploads no longer inflate metrics. The Cleanups
   metric additionally filters event='complete' to exclude the new
   retry-incomplete precursor rows.

2. New dashboard panels: Equity/uPnL/Available trend (consuming
   cycle_summary.account, live-only), Data Freshness (max(_timestamp)),
   Rejections & Error Signals, and Stream Health / Reconnects.

3. Precursor eprintln signals now emit structured stdout JSON events in
   JSON mode so the existing stdout ingest uploads them: cancel-retry
   failures (recovery.rs, action=maker_cleanup event=retry_incomplete) and
   reconciliation-snapshot failures (runtime.rs, action=position_reconciliation
   event=snapshot_failed), both severity=warning. Non-JSON output keeps the
   human-readable stderr line, matching the existing emit_* pattern.

4. Ingest checkpoint hardening: checkpoints now record file identity
   (inode + size) and reset to zero on rotation/truncation; a corrupt state
   file auto-resets instead of wedging --follow at startup; and the
   checkpoint map is bounded (LRU eviction) so openobserve-uploaded.json no
   longer grows unbounded. event_id idempotency remains the dedup safety net.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@wjllance wjllance force-pushed the fix-224-dashboard-distinct-panels branch from 7606f0d to 981bf83 Compare July 13, 2026 08:26
@wjllance wjllance merged commit a528397 into main Jul 13, 2026
7 checks passed
@wjllance wjllance deleted the fix-224-dashboard-distinct-panels branch July 13, 2026 08:43
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[P1][Monitoring] dashboard 计数改 DISTINCT、补 equity/新鲜度面板、stderr 前兆信号上传

2 participants