chore: add gitleaks pre-commit hook#3
Merged
Merged
Conversation
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
cipher813
added a commit
that referenced
this pull request
May 1, 2026
* feat(ci): wire deploy.yml + deploy-infrastructure.yml into system-wide changelog Adds a final step to both deploy workflows that calls the append-changelog composite action in alpha-engine-docs. Each successful (or failed) deploy now emits one JSON to s3://alpha-engine-research/changelog/. Two distinct entries per merge that touches both surfaces: - deploy.yml → Phase 2 Lambda image rebuild + alias bump - deploy-infrastructure.yml → SF + CF stamp re-deploy Distinguished by the deploy_workflow field on each entry, so the materialized CHANGELOG.md can show both as separate items under the same SHA. Uses if: always() + ternary on job.status so failed deploys also register in the log — the failure signal is itself a useful provenance record. Companion: alpha-engine-docs PR #3 (composite action + aggregator), alpha-engine-data PR #120 (IAM grant — already merged). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(orchestration): SNS→S3 changelog incident mirror Lambda Adds a small Lambda subscribed to the alpha-engine-alerts SNS topic that mirrors every alert as one JSON entry under s3://alpha-engine-research/changelog/incidents/. Closes the event- mining loop alongside the deploy-side log: now both "what shipped" and "what failed" feed the same time-ordered changelog. Why The 2026-05-01 weekday SF timeout cascade is the canonical example. The deploy log records the 4 PRs that fixed it, but it never captured the original SNS alert email at 06:01 PT — the failure event itself. With this Lambda, that alert would have landed at changelog/incidents/2026/05/01T13-01-XX_alpha-engine-alerts_*.json with full subject + body, queryable months later for retro mining ("show me every SF failure incident this quarter"). Resources added (4) - ChangelogIncidentMirrorRole — minimal: PutObject scoped to changelog/incidents/* + AWSLambdaBasicExecutionRole for logs. - ChangelogIncidentMirrorFunction — python3.12, arm64, 256 MB, 30s timeout. Inline ZipFile (~50 lines). Reads SNS Records, builds a JSON entry, S3 PutObject. No-ops cleanly on malformed timestamps (falls back to "now"). - ChangelogIncidentMirrorSubscription — SNS subscription on AlertsTopic with Protocol: lambda. - ChangelogIncidentMirrorPermission — Lambda::Permission letting SNS invoke the function. Schema (matches the deploy-side action's event_type discriminator) { "ts_utc": ..., "event_type": "incident", "source": "alpha-engine-alerts", "subject": "...", "summary": "...", // first 240 chars of subject or message line 1 "details": "...", // full message body "sns_message_id": "...", "topic_arn": "..." } Apply state Already applied live via aws cloudformation execute-change-set; smoke-tested with one SNS publish — entry landed at s3://alpha-engine-research/changelog/incidents/2026/05/01T15-52-57_* within 2s, schema validated, then cleaned up. This PR is the codification of the source-of-truth template. Companions - alpha-engine-docs PR #5 (event_type schema + aggregator support) - Future: flow-doctor S3 notifier, manual CLI helper. Note on template description The template's docstring says "Does NOT manage Lambda functions or IAM roles." Strictly we now manage one of each — narrow exception for the SNS-mirror because it's tightly coupled to AlertsTopic defined here. Not updating the doc this commit; will revisit if a second exception lands. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
cipher813
added a commit
that referenced
this pull request
May 25, 2026
…hase 0.2) (#308) * feat(cost-telemetry): wire news event extraction + lib v0.32→v0.33 (Phase 0.2) Closes the largest previously-untracked LLM cost slice in the system (~$20–60/mo, the dominant ~20–60% of monthly Anthropic spend per the Phase 0 audit at ``alpha-engine-docs/private/prompt-caching-investigation-260525.md`` §1.1). News-article event extraction in ``collectors/nlp/event_extraction.py:167`` fires 100–300 Haiku calls per RAGIngestion run; this PR records every response's tokens + server-tool fees into a per-run JSONL flushed once to S3 at the canonical research-side cost-raw partition. **Zero-coupling pattern:** new ``rag/pipelines/_cost_telemetry.py`` provides ``wrap_client_for_cost_telemetry(client, buffer)`` — proxy that records every ``messages.create()`` response into a buffer without changing the response shape returned. ``AnthropicEventExtractor`` is UNCHANGED — telemetry composes at the client-construction layer in ``_run_nlp`` rather than polluting the extractor with cost-tracking concerns. **Single S3 chokepoint:** rows land at ``s3://alpha-engine-research/decision_artifacts/_cost_raw/{date}/{date}/data:news_event_extraction.jsonl`` — same partition the research-side ``aggregate_costs.py`` already scans. The daily parquet now sums data's rows alongside research's under one ``by_agent_id`` breakdown; the dashboard cost panel (Phase 0.3) will show every site in one view. **Buffered + flushed once** rather than per-call to keep S3 PutObject volume sane: 100–300 calls per run → 1 PutObject at end-of-pipeline. **Fail-loud at flush** per ``[[feedback_no_silent_fails]]``: S3 PutObject failure raises ``CostBufferFlushError`` and fails the pipeline — silent miss on the dominant cost slice would defeat the Phase 0 visibility goal. Per-call recording failures (malformed response shape) are logged but do NOT propagate — the event extractor's primary deliverable must survive a cost-telemetry hiccup. **Lib pin v0.32.0 → v0.33.0** in requirements.txt + Dockerfile to consume the lifted ``alpha_engine_lib.cost.record_anthropic_call`` (alpha-engine-lib #69, the SOTA chokepoint that data + executor are consumers #2 + #3 of after morning-signal originated the pattern). **Tests:** new ``tests/test_news_cost_telemetry.py`` covers buffer record/flush/empty/error paths + proxy passthrough + per-call recording failure isolation + factory naming convention. 9 new tests, suite 1480 → 1489 passing, zero regressions. Closes ROADMAP Phase 0.2 row "News event extraction (data, $20–60/mo, dominant slice)" — first production exercise will land on the next Saturday SF (2026-05-30) when RAGIngestion fires. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * test(news-cost-telemetry): drop moto dep, use in-memory S3 mock CI failure on PR #308: ``ModuleNotFoundError: No module named 'moto'`` — this repo's CI installs only ``requirements.txt + pytest`` (no dev-extras file), and the rest of the suite avoids moto by using a minimal in-memory S3 mock (``_InMemoryS3`` in ``tests/test_news_aggregates.py``). Mirror that pattern here so the cost-telemetry tests run cleanly in the unmodified CI environment. Functional coverage unchanged — all 9 tests still pass. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
.pre-commit-config.yamlwith gitleaks hook for pre-commit secret scanning.gitignorefor gitleaks config and baseline files🤖 Generated with Claude Code