Skip to content

docs: the sync runs from the producer, and the stale local paths that block it - #15

Merged
mspinola merged 1 commit into
mainfrom
claude/sync-runs-from-the-producer
Aug 5, 2026
Merged

mspinola merged 1 commit into
mainfrom
claude/sync-runs-from-the-producer

Conversation

@mspinola

@mspinola mspinola commented Aug 5, 2026

Copy link
Copy Markdown
Owner

Docs only. States where the sync must run from, and records a pre-existing blocker I found while trying to run it.

1. The sync runs from the producer

push_data_cache_to_server.sh must run on the Windows/Norgate box. That is where the prices and contract specs exist, so it is where every payload has to be built before it can be shipped. Run it from a machine that merely reads a synced store and it pushes a copy of an older sync — silently, since rsync has no opinion about which direction is authoritative.

That was implicit in "Norgate is Windows-only" and is now stated in the header, because the fourth payload made it ambiguous: crowdmon's damage panel is derived output that any machine with the store can regenerate, so it does not inherit the vendor argument the other three payloads have. It still has to be built upstream of the sync.

server-side/README.md gains the Windows invocation, since crowdmon/bin/publish_damage.sh is bash and the launchd agent beside it is macOS — neither runs there:

set COTDATA_STORE=...\cotdata_store
set CROWDMON_STORE=...\crowdmon_store
.venv\Scripts\python bin\publish_damage.py

Publish before the sync, or it ships last week's panel beside this week's prices. The /damage page reports that as a staleness banner rather than an error, which is correct behaviour and also means nobody finds out from the dashboard.

One overstatement corrected

An earlier draft implied crowdmon needs Windows. It does not — it runs anywhere the store is readable, and builds the full 47-market panel on macOS today. The constraint is that Norgate, which is the only vendor supplying every price tier crowdmon needs, talks to a locally installed Windows application rather than an API. server-side/README.md now says the rule is "built upstream of whatever ships it", not "built on Windows".

2. Not fixed here: this script has been unable to run since #12

Found by trying to dry-run it. This is pre-existing and unrelated to the crowdmon payload.

PR #12 ("Route local runtime files to a shared, untracked workspace dir") moved runtime state out of the repo. The push script still points into it:

script expects actual location state
./data/cot_data.db .local-state/cot-analyzer/cot_data.db absent → preflight refuses, exit 1
./data_cache/ .local-state/cot-analyzer/data_cache/ repo copy has 48 files, last written Jul 28; live has 50

The DB one is why a dry run aborts today. The data_cache one is quieter and worse: had the preflight passed, the script would have rsynced a week-stale cache over the server's and said nothing about it.

I have not repointed those two paths, because it changes what lands on production and I would be inferring the intent from a PR title rather than from a stated one. Happy to do it as a follow-up with a dry run you can read before anything moves.

Companions

Testing

170 passed; ruff clean; bash -n clean on the script.

🤖 Generated with Claude Code

… block it

The push script must run on the Windows/Norgate box: that is where the prices and
contract specs exist, so it is where every payload has to be built before it can be
shipped. Run from a machine that merely READS a synced store and it pushes a copy of
an older sync. Stated in the header rather than left to be inferred from the fact
that Norgate is Windows-only.

Same for the new fourth payload: crowdmon/bin/publish_damage.py runs on that machine,
before this script, or the sync ships last weeks panel beside this weeks prices.
Nothing enforces the ordering.

server-side/README gains the Windows invocation, since bin/publish_damage.sh is bash
and the launchd agent beside it is macOS. Neither runs there.

NOT fixed here, because it changes what lands on production and wants a human: this
script has been unable to run since #12 routed runtime state out of the repo. It
still preflights ./data/cot_data.db, which now lives in
.local-state/cot-analyzer/, so it exits 1; and ./data_cache is a Jul 28 copy of a
cache that now lives there too, so had the preflight passed it would have pushed a
stale cache silently.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@mspinola
mspinola merged commit 6b5fcbb into main Aug 5, 2026
3 checks passed
@mspinola
mspinola deleted the claude/sync-runs-from-the-producer branch August 5, 2026 02:03
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant