diff --git a/CLAUDE.md b/CLAUDE.md index be5f7b1..da2b590 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -4,7 +4,7 @@ Project guidance for Claude Code sessions in `cgfixit/Veeam-HealthCheck-Simplifi ## Project Overview -Single-file Python CLI (`vhc_simplifier.py`, ~770 lines) that processes Veeam Health Check exports from VBR servers (v12.3.2 and v13) into remediation artifacts. Input comes from running https://vee.am/vhc2 on Windows Server. +Single-file Python CLI (`vhc_simplifier.py`, v4.1.0) that processes Veeam Health Check exports from VBR servers (v12.3.2 and v13) into remediation artifacts. Input comes from running https://vee.am/vhc2 on Windows Server. ## Commands @@ -15,7 +15,7 @@ pip install -r requirements.txt # Run demo mode python vhc_simplifier.py --demo -# Run all tests (219 tests, 90%+ coverage) +# Run all tests (260 collected; CI fails under 80% coverage) python -m pytest tests/ -v # Run with coverage @@ -35,7 +35,7 @@ python -m py_compile vhc_simplifier.py - **Single-file CLI** — `vhc_simplifier.py` is the entire application. Do not split into a package unless explicitly asked. - **Fault-isolated analyzers** — each `analyze_*()` function runs inside `_run_analyzer()`, which catches exceptions per-analyzer. A failing analyzer records an error and skips, never aborts the run. -- **Adapters** — `_safe_load_csv()` and `_safe_load_json()` handle encoding quirks (UTF-8 BOM, encoding_errors replacement). The rest of the pipeline works on pandas DataFrames. +- **Adapters** — `_safe_load_csv()` and `_safe_load_json()` handle encoding quirks (UTF-8 BOM, encoding fallback, strict decode). The rest of the pipeline works on pandas DataFrames. - **Enrichment** — `enrich_findings()` pattern-matches findings to PowerShell commands. `_ps_quote()` guards against injection. Control characters trigger `REFUSED`. - **Artifact writers** — `_write_artifact()` wraps each output file write in its own try/except. @@ -44,9 +44,9 @@ python -m py_compile vhc_simplifier.py - **Python 3.12+** — do not use features removed before 3.12 or added after 3.13. - **Preserve `-WhatIf` defaults** on all mutating PowerShell verbs. Never remove without explicit request. - **No secrets in code** — Salesforce credentials come from env vars only. Never log or hardcode tokens. -- **Slack webhook validation** — only `https://hooks.slack.com/...` and `https://hooks.slack-gov.com/...` are accepted. +- **Slack webhook validation** — only `https://hooks.slack.com/services/...` and `https://hooks.slack-gov.com/services/...` are accepted. - **Analyzer signatures** — analyzers take DataFrames directly: `analyze_jobs(jobs_df, sessions_df)`, `analyze_security(sec_df)`, `analyze_repositories(repo_df)`, `analyze_malware(malware_df)`. They do NOT take a `result` parameter. -- **Encoding robustness** — always use `encoding_errors="replace"` for CSV loading. Strip UTF-8 BOM from JSON with `.lstrip("")`. +- **Encoding robustness** — CSV loading tries `_candidate_encodings` with `encoding_errors="strict"` and rejects NUL-decoded files. `_read_text()` strips a UTF-8 BOM (`\ufeff`) before JSON parse. ## Test Structure @@ -63,17 +63,18 @@ Tests live in `tests/` with shared fixtures in `conftest.py`. The conftest provi ## CI Workflows -Six GitHub Actions workflows in `.github/workflows/`: -- `ci.yml` — lint + test matrix (Python 3.12/3.13, ubuntu/windows), 80% coverage threshold +Seven GitHub Actions workflows in `.github/workflows/`: +- `ci.yml` — lint + test matrix (Python 3.12/3.13, ubuntu/windows), 80% coverage threshold, Docker smoke - `ruff.yml` — Ruff lint + format - `gitleaks.yml` — secret scanning - `devskim.yml` — Microsoft security linting (tests/ excluded via `ignore-globs`) - `codeql.yml` — GitHub static analysis -- `dependency-review.yml` — supply chain checks +- `dependency-review.yml` — supply chain checks (pull requests only) +- `trivy.yml` — filesystem + Docker image vulnerability scanning ## Style -- Use `ruff` for linting and formatting. No custom config needed — defaults work. +- Use `ruff` for linting and formatting. Config lives in `pyproject.toml` (`target-version = "py312"`, `line-length = 110`). - Minimal comments. The code should be self-documenting. - Prefer stdlib and existing dependencies over adding packages. - Keep changes in the fewest files possible. diff --git a/README.md b/README.md index 325edd8..55cb112 100644 --- a/README.md +++ b/README.md @@ -1,6 +1,6 @@ # Veeam Health Check Simplifier -**A security-focused VBR pipeline that ingests [Veeam Health Check](https://vee.am/vhc2) exports from Veeam Backup & Replication (VBR) servers, analyzes them for optimizations, fixes, security hardening, compliance gaps, and generates safe, reviewable remediation artifacts.** +**v4.1.0** — a security-focused VBR pipeline that ingests [Veeam Health Check](https://vee.am/vhc2) exports from Veeam Backup & Replication (VBR) servers, analyzes them for optimizations, fixes, security hardening, compliance gaps, and generates safe, reviewable remediation artifacts. --- @@ -39,7 +39,7 @@ This project takes those raw exports and transforms them into actionable remedia │ vhc_simplifier.py │ │ │ │ 1. Load exports (CSV or JSON) │ - │ 2. Analyze across 5 domains │ + │ 2. Analyze 4 sections (5 inputs) │ │ 3. Enrich with remediation cmds │ │ 4. Generate artifacts │ │ 5. (Optional) Push to SF/Slack │ @@ -54,9 +54,8 @@ This project takes those raw exports and transforms them into actionable remedia ### Analysis Pipeline 1. **Load** — `_safe_load_csv()` / `_safe_load_json()` with UTF-8 BOM stripping, Windows-oriented encoding fallback, corrupt-decode guards, and empty-file guards -2. **Analyze** — fault-isolated analyzers (`_run_analyzer()` wrapper catches exceptions per-analyzer, never aborts the run): - - `analyze_jobs()` — retention, encryption, schedule gaps - - `analyze_sessions()` — recent failures and warnings +2. **Analyze** — four fault-isolated analyzers (`_run_analyzer()` wrapper catches exceptions per-analyzer, never aborts the run): + - `analyze_jobs(jobs_df, sessions_df)` — retention, encryption, and recent session failures (there is no separate `analyze_sessions()`) - `analyze_security()` — best-practice compliance status - `analyze_repositories()` — immutability support gaps - `analyze_malware()` — infected/suspicious scan results @@ -73,7 +72,7 @@ This project takes those raw exports and transforms them into actionable remedia - **Multi-format input** — CSV (default, from [vee.am/vhc2](https://vee.am/vhc2)) or JSON (VBR REST API exports) - **Demo mode** — runs instantly with embedded real-world sample data; no input files required -- **Secure PowerShell output** — all mutating commands (`Set-`, `New-`, `Remove-`, etc.) include `-WhatIf` by default +- **Secure PowerShell output** — mutating cmdlets (`Set-`, `New-`, `Remove-`, etc.) include `-WhatIf` by default; several remediations (storage encryption, retention, immutability) are review-only comments, not mutating cmdlets - **PS injection prevention** — object names with control characters are refused, not interpolated - **Salesforce integration** — creates Tasks on an Account record for High/Medium findings - **Slack notifications** — posts a severity summary to an incoming webhook @@ -95,14 +94,22 @@ This project takes those raw exports and transforms them into actionable remedia ## Requirements -- Python 3.12+ (tested on 3.12 and 3.13) +- Python 3.12+ (tested on 3.12 and 3.13); package version **4.1.0** (`pyproject.toml`) - `pandas` (core dependency) - `simple-salesforce` *(optional — only for `--sf-account-id`)* - `httpx` *(optional — Slack fallback uses stdlib `urllib` if absent)* -- `pytest`, `pytest-cov` *(dev/testing only)* +- `pytest`, `pytest-cov`, `ruff` *(dev/testing only)* ```bash +# Core + optional integrations + dev tools (matches requirements.txt) pip install -r requirements.txt + +# From a clone, slimmer extras defined in pyproject.toml: +# pip install -e ".[salesforce]" # simple-salesforce +# pip install -e ".[slack]" # httpx +# pip install -e ".[integrations]" # both +# pip install -e ".[dev]" # pytest, pytest-cov, ruff +# After `pip install -e .`, the console script `vhc-simplifier` calls main(). ``` --- @@ -180,8 +187,7 @@ root cron daemon inside the image. | `--input-format` | `csv` | Input format: `csv` or `json` | | `--demo` | off | Use embedded sample data (no files needed) | | `--no-artifacts` | off | Skip writing `.md`/`.ps1`/`.json` files | -| `--quiet` | off | Suppress console report | -| `--verbose` | off | Enable verbose/debug logging | +| `--quiet` | off | Suppress the default console report (`run_healthcheck(verbose=)` is the Python API; there is no `--verbose` flag) | | `--sf-account-id` | — | Salesforce Account ID for Task creation | | `--sf-username` | — | SF username (prefer `SF_USERNAME` env var) | | `--sf-password` | — | SF password (prefer `SF_PASSWORD` env var) | @@ -202,7 +208,7 @@ These files are produced by running the [Veeam Health Check script](https://vee. | Repositories | `localhost_Repositories.csv` | `localhost_Repositories.json` | | Malware | `localhostmalware_events.csv` | `localhostmalware_events.json` | -All files are optional — missing files are logged and skipped without aborting. +All files are optional — missing files are logged and skipped without aborting. Exact `localhost_*` names are the VHC defaults; hostname-prefixed variants (for example `VBR01_Jobs.csv`) are also discovered. --- @@ -221,7 +227,7 @@ python vhc_simplifier.py --demo --sf-account-id 001XXXXXXXXXXXX ### Slack -Posts a severity summary to a Slack incoming webhook. Validates that the webhook URL uses HTTPS and targets `hooks.slack.com` or `hooks.slack-gov.com`: +Posts a severity summary to a Slack incoming webhook. Accepts only `https://hooks.slack.com/services/...` and `https://hooks.slack-gov.com/services/...`: ```bash python vhc_simplifier.py --demo \ @@ -251,16 +257,16 @@ docker compose run --rm vhc-simplifier \ --slack-webhook https://hooks.slack.com/services/T00/B00/xxx ``` -Salesforce credentials (`SF_USERNAME`, `SF_PASSWORD`, `SF_TOKEN`) are passed through from the host environment or a `.env` file — never bake them into the image. Neither `./vhc-exports` nor `./output` should ever be committed; both are covered by `.gitignore`. +Salesforce credentials (`SF_USERNAME`, `SF_PASSWORD`, `SF_TOKEN`) are passed through from the host environment or a `.env` file — never bake them into the image. The `Dockerfile` installs `pandas` only: Slack posting uses stdlib `urllib` when `httpx` is absent, and Salesforce Task creation needs `simple-salesforce`, which is not in the image. Neither `./vhc-exports` nor `./output` should ever be committed; both are covered by `.gitignore`. --- ## Testing -The test suite covers 219 tests across 6 files with 90%+ code coverage: +The test suite collects **260** tests across 5 modules plus shared fixtures in `conftest.py` (CI `--cov-fail-under=80`): -| Test File | Tests | Coverage | -|---|---|---| +| Test File | Purpose | +|---|---| | `test_vhc_simplifier.py` | Core helpers, analyzers, loaders, enrichment, artifact writers, integration runs | | `test_coverage_gaps.py` | Edge cases: NaN handling, empty DataFrames, encoding, type coercion | | `test_vbr_server_simulation.py` | VBR v12.3.2 and v13 mock server simulation with realistic export data | @@ -283,11 +289,11 @@ python -m pytest tests/test_vbr_server_simulation.py -v ## CI/CD Workflows -Six GitHub Actions workflows run on every push and pull request to `main`: +Seven GitHub Actions workflows live in `.github/workflows/`. CI, Ruff, Gitleaks, DevSkim, CodeQL, and Trivy run on push and pull request to `main` (Gitleaks, DevSkim, CodeQL, and Trivy also run weekly). Dependency Review runs on pull requests only. | Workflow | File | Purpose | |---|---|---| -| **CI** | `ci.yml` | Lint + test matrix (Python 3.12/3.13 on ubuntu + windows), 80% coverage threshold | +| **CI** | `ci.yml` | Lint + test matrix (Python 3.12/3.13 on ubuntu + windows), 80% coverage threshold, Docker image smoke test | | **Ruff** | `ruff.yml` | Python lint and format checks | | **Gitleaks** | `gitleaks.yml` | Secret scanning (push, PR, weekly) | | **DevSkim** | `devskim.yml` | Microsoft security anti-pattern detection (push, PR, weekly) | @@ -321,7 +327,7 @@ Input (CSV / JSON / --demo) _run_analyzer() <- fault isolation wrapper (records errors, never aborts) | v - analyze_*() <- pure functions, domain-scoped (jobs, sessions, security, repos, malware) + analyze_*() <- four pure functions (jobs+sessions, security, repos, malware) | v enrich_findings() <- pattern-matched remediation + PS injection guard (_ps_quote) diff --git a/SECURITY.md b/SECURITY.md index 7b9074f..159a477 100644 --- a/SECURITY.md +++ b/SECURITY.md @@ -15,7 +15,7 @@ This project is a single-script utility (`vhc_simplifier.py`) and does not follo ## Scope -This tool parses and summarizes Veeam Health Check HTML/JSON reports. Security considerations relevant to this project include: +This tool parses and summarizes Veeam Health Check CSV/JSON exports. Security considerations relevant to this project include: - **Path traversal / arbitrary file read** — malicious input file paths passed to the script - **Unsafe deserialization** — processing of crafted/malformed Veeam report files