fix: OSD Global-tenant import + dropped report files with glob metacharacters; validate dev stack on OpenSearch 3.x with PostgreSQL

[seanthegeek]

Started as the OSD Global-tenant import bug; auditing the rest of the dev stack surfaced real library bugs, a shipped-dashboard mapping conflict, and several dev-stack gaps. Everything was validated end-to-end against a freshly rebuilt stack (OpenSearch 3.6.0 + OSD 3.5).

Bug fixes (released behavior)

1. OSD saved objects imported into the wrong tenant. dashboard-dev-bootstrap.sh sent securitytenant: global_tenant. The security plugin reads that as a tenant name, and global_tenant is a sample custom tenant in the demo config — not the Global tenant (token global). The import landed in a phantom tenant; Global looked empty. Fixed to global. (Shipped 9.10.3–9.11.2.)

2. Report files with glob metacharacters in their names were silently dropped. The CLI ran every file argument through glob(), which treats [, ], *, ? as patterns. A literal [Netease DMARC Failure Report] Rent Reminder.eml matched nothing and never reached the parser. Files that exist on disk are now used literally; only non-existent paths are globbed. (This is why "4 failure samples → only 3 in the dashboard".)

3. OSD mapping conflict on the aggregate org_email field. The shipped opensearch_dashboards.ndjson froze a cached field-list where org_email was a text/object conflict (plus stale org_email.#text* subfields) — from a cluster that indexed a langAttrString email dict before the parser unwrapped it. org_email is Text() and the parser now unwraps dict emails, so live data is clean; cleared the frozen conflict + artifacts. (Shipped in 9.11.2.)

Bug fixes 1–2 have regression tests in tests/test_cli.py.

Completing the (unreleased) PostgreSQL feature

• Added postgresql to _KNOWN_SECTIONS so PARSEDMARC_POSTGRESQL_* env vars and their _FILE Docker-secret variants resolve like every other backend (was silently ignored). New in 10.0.0, so documented under the PostgreSQL enhancement, not Bug fixes. Has a test.

Dev-stack audit follow-ups (bootstrap + compose)

• OpenSearch 2.x → 3.x (unsupported cross-major with OSD 3.x); validated on 3.6.0.
• PostgreSQL added to the dev stack: service + bootstrap wiring (wait, seed via PARSEDMARC_POSTGRESQL_*, RESEED wipe, Grafana grafana-postgresql-datasource, PG dashboard import). Seeding is gated on psycopg being importable — parsedmarc exit(1)s during client init if a configured backend can't load, which would otherwise zero every backend.
• Grafana password env fix: container got GRAFANA_PASSWORD, which Grafana ignores; it reads GF_SECURITY_ADMIN_PASSWORD.

Docs

• PostgreSQL ships a premade Grafana dashboard, so it now sits on the "premade dashboards" features bullet (README + docs/source/index.md).

Validation (rebuilt stack)

Check	Result
OSD import → global tenant (OS 3.6.0)	26 objects / 3 dashboards
Aggregate index pattern org_email	text, no conflict
Failure docs indexed (dmarc_f*)	4 (Netease/cardinal.com included)
PostgreSQL rows	aggregate 11, failure 3, smtp_tls 4
Grafana → PostgreSQL datasource health	OK – Database Connection OK
Full test suite	628 passed

🤖 Generated with Claude Code

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 85.58%. Comparing base (411f5a8) to head (4b5fc72).
✅ All tests successful. No failed tests found.

Additional details and impacted files

@@            Coverage Diff             @@
##           master     #781      +/-   ##
==========================================
+ Coverage   85.54%   85.58%   +0.03%     
==========================================
  Files          17       17              
  Lines        4628     4633       +5     
==========================================
+ Hits         3959     3965       +6     
+ Misses        669      668       -1     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.