Skip to content

fix(archive): persist consolidated_report to S3 — closes 2-month dashboard archive stale#208

Merged
cipher813 merged 2 commits into
mainfrom
fix/persist-consolidated-report-to-s3
May 20, 2026
Merged

fix(archive): persist consolidated_report to S3 — closes 2-month dashboard archive stale#208
cipher813 merged 2 commits into
mainfrom
fix/persist-consolidated-report-to-s3

Conversation

@cipher813
Copy link
Copy Markdown
Owner

Summary

archive_writer was building the weekly consolidated brief into state.consolidated_report, handing it off to email_sender for the morning email, and then dropping it. ArchiveManager.save_consolidated_report() existed but had no caller anywhere in the codebase — the persistence wire to s3://alpha-engine-research/consolidated/{date}/morning.md was lost.

Evidence of the wiring loss: last morning.md write to S3 was 2026-03-16. Every week since (~10 runs) emailed the brief to the user but never persisted it. The dashboard's Research Briefing Archive page was correctly showing the last actually-persisted artifact — that's why "Latest" displayed 2026-03-16 while EOD emails kept landing in the same date partition.

$ aws s3 ls s3://alpha-engine-research/consolidated/2026-05-20/
2026-05-20 13:27:38   6849 eod.html       # ← still written
                                          # ← morning.md missing

Fix

3 lines in archive_writer right next to write_signals_json, same try/except shape so a save failure doesn't block the rest of the archive pipeline.

consolidated = state.get("consolidated_report", "") or ""
if consolidated:
    try:
        am.save_consolidated_report(run_date, consolidated)
    except Exception as e:
        logger.error("Failed to save consolidated_report: %s", e)

Tests

  • test_save_consolidated_report_writes_morning_md_to_s3 — pins the producer side: save_consolidated_report("2026-05-20", brief) writes one put_object call with Key=consolidated/2026-05-20/morning.md and the brief body.
  • test_archive_writer_wires_save_consolidated_report — structural pin via inspect.getsource(archive_writer). If the wiring is dropped again, this fails at CI time instead of silently staling the archive surface for two months.
  • 1375/1375 suite green.

Backfill

Historical 2026-03-16 → 2026-05-17 gap can't be backfilled — those briefs were emailed only, not retained anywhere else. Next Saturday SF research run (2026-05-23) populates the first new morning.md and the archive page surfaces it the next time the dashboard refreshes its cache (TTL 15 min).

🤖 Generated with Claude Code

cipher813 and others added 2 commits May 20, 2026 15:52
…doesn't stale

archive_writer was building the consolidated morning brief into
`consolidated_report` state, passing it to email_sender for the
weekly email, and then dropping it on the floor —
ArchiveManager.save_consolidated_report() existed but had no caller
for ~2 months. Last `consolidated/{date}/morning.md` write was
2026-03-16; from then through 2026-05-20 the dashboard's Research
Briefing Archive page correctly showed "Latest 2026-03-16" because
that was genuinely the last persisted artifact in S3.

The wiring loss almost certainly happened in the mid-March LangGraph
refactor — the persistence call lived in the legacy pre-graph
path and didn't get carried into the new archive_writer node.

Fix: 3 lines in archive_writer right next to write_signals_json,
same try/except shape so a save failure doesn't block the rest of
the archive pipeline.

Regression tests:
- save_consolidated_report writes consolidated/{date}/morning.md
  to S3 with the brief body.
- archive_writer's source contains `save_consolidated_report` —
  structural pin so removing the call again fails CI instead of
  silently staling the archive surface for another two months.

Dashboard reads from `consolidated/` so no consumer change needed;
the next Saturday SF research run will repopulate from 2026-05-23
onward. Historical 2026-03-16 → 2026-05-17 gap can't be backfilled
(the briefs were emailed, not retained anywhere else).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@cipher813 cipher813 merged commit dcf16ea into main May 20, 2026
1 check passed
@cipher813 cipher813 deleted the fix/persist-consolidated-report-to-s3 branch May 20, 2026 23:06
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant