Skip to content
This repository was archived by the owner on Aug 8, 2026. It is now read-only.

docs: which machine publishes, and why only one can write a store this package reads - #67

Merged
mspinola merged 2 commits into
mainfrom
claude/publish-runs-on-the-producer
Aug 5, 2026
Merged

mspinola merged 2 commits into
mainfrom
claude/publish-runs-on-the-producer

Conversation

@mspinola

@mspinola mspinola commented Aug 5, 2026

Copy link
Copy Markdown
Owner

Docs only. Two corrections, one of them to a mistake this session made and shipped.

1. The production publish runs on the producer, not on this Mac

cot-analyzer/server-side/README.md already said the panel is built on the Windows/Norgate producer. I then installed a launchd agent on the Mac and described it as the schedule. Both are true of different things and nothing said which.

The Linux server's sync originates on the Windows box, so a panel built downstream of that sync is one the sync never sees. bin/publish_damage.sh and the plist beside it are macOS/Linux, and what they feed is a local cot-analyzer — genuinely useful, and not a deploy.

The agent stays installed. Recording that it was installed against a document already saying otherwise is worth more than quietly deleting it.

Windows has no bash, so the production path is the portable driver:

set COTDATA_STORE=...\cotdata_store
set CROWDMON_STORE=...\crowdmon_store
.venv\Scripts\python bin\publish_damage.py

And the ordering constraint nothing enforces, now stated in all three places rather than left to be discovered: publish before the Windows → Linux sync, or the sync ships last week's panel beside this week's prices.

The first wording was too strong, and is corrected here

I first wrote "the panel that reaches production has to be built there", which reads as a capability limit on this package. It is not one. crowdmon runs anywhere the store is readable, macOS included, on the full price-dependent chain — a local run builds all 47 markets with beta attached and both trigger sides, and §D13's figures were measured that way.

The real rule is narrower: the panel must be built upstream of whatever ships it. Today that is the Windows box because the sync originates there. If the sync origin moves, the publish moves with it and no code changes.

Also worth separating, because it decides the fallback: the panel is the one payload with no producer-origin argument of its own. Prices are vendor data and can only come from the subscription machine; the panel is derived output that any machine with the store regenerates identically.

2. §D14 — only the Norgate producer can write a store this package can read

A tier fact, not an OS fact, and the three machines fail for three different reasons, which is why lumping them under "Windows-only" was misleading.

box Python ≥3.10 Norgate can produce a store this package reads
Windows yes yes yes
macOS yes, runs 3.11 under uv and builds the full panel today no no
Linux server no, runs 3.9 no no

Three facts that were each already written down, composed here because nothing had composed them:

  1. notional refuses anything but unadj, riskunits anything but propadj, and propadj is derived on read from unadj + backadj — so both stored tiers are a precondition. ContractMaster.coverage() already encodes it.
  2. Norgate is Windows-only by mechanism, not licence: norgatedata talks to a locally installed updater application rather than an API.
  3. crucible-stack ADR-0007 scopes databento to exactly one series per symbol, backadj, and says its coverage should not broaden.

Composed: a databento-backed futures store cannot feed this package at all. The refusal is a raise and not a warning because the error survives a spot check — backadj percent vol is 201x too high for soybeans and 0.47x for gold, which never goes negative.

So the sentence to carry is not "crowdmon needs Windows". It is "crowdmon needs a full-tier store, and today exactly one machine can write one."

The trap this creates for ADR-0007 step 2

marketdata has no futures provider, so step 2 writes one. If it ships databento-first, or ships Norgate storing only backadj to match the server's needs, this package stops working — as a raise inside a weekly scheduled job.

Work order filed as cotdata/docs/handoffs/2026-08-04-adr7-step2-price-producer-split.md (mspinola/cotdata#101), which carries that requirement in §4 where its author will be reading.

Testing

Docs, a shell comment and a plist comment. 560 passed / 95 skipped from a worktree, unchanged; ruff clean; plutil -lint clean on the plist. The four pinned suite counts do not move.

🤖 Generated with Claude Code

mspinola and others added 2 commits August 4, 2026 21:29
cot-analyzer/server-side/README.md already said the panel is built on the
Windows/Norgate producer. I then installed a launchd agent on the Mac and treated
it as the schedule. Both are true of different things and the docs did not say so.

The sync to the Linux server originates on the Windows box, so a panel built
anywhere else is one the sync never sees. Windows has no bash, so the production
path is the portable driver, python bin/publish_damage.py, and NOT the .sh wrapper
or the plist beside it. Those feed a local cot-analyzer, which is genuinely useful
and is not a deploy.

The ordering constraint nothing enforces, stated in all three places rather than
left to be discovered: publish BEFORE the Windows -> Linux sync, or the sync ships
last weeks panel beside this weeks prices.

The agent stays installed. Recording that it was installed against a document that
already said otherwise is more useful than quietly deleting it.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
A tier fact, not an OS fact, and the three machines fail for three different
reasons which is why lumping them under "Windows-only" was misleading.

notional refuses anything but unadj, riskunits anything but propadj, and propadj
is DERIVED ON READ from unadj + backadj, so both stored tiers are a precondition.
ContractMaster.coverage() already encodes it. Norgate is the only vendor supplying
all tiers and is Windows-only by mechanism rather than licence: norgatedata talks
to a locally installed updater application, not an API. And crucible-stack ADR-0007
scopes databento to exactly one series per symbol, backadj, and says its coverage
should not broaden.

Composed, those three mean a databento-backed futures store cannot feed this
package at all. Nothing had composed them.

macOS has no Python problem and the table says so: it runs 3.11 under uv and built
the full 47-market panel today, including every price-dependent rung. D13 was
measured there. Its blocker is one vendor application that does not exist for the
platform.

The trap this creates for ADR-0007 step 2: marketdata has no futures provider, so
step 2 WRITES one. If it ships databento-first, or Norgate storing only backadj to
match the servers needs, this package stops working as a raise inside a weekly
scheduled job. Work order filed as cotdata/docs/handoffs/2026-08-04-adr7-step2-price-producer-split.md.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@mspinola
mspinola merged commit 61fd5d6 into main Aug 5, 2026
5 checks passed
@mspinola
mspinola deleted the claude/publish-runs-on-the-producer branch August 5, 2026 02:03
mspinola added a commit that referenced this pull request Aug 5, 2026
…he server

docs/examples/windows/run-publish.cmd and push-panel.cmd, following cotdatas
convention exactly: copy out of the repo into a scheduler dir, fill the
REPLACE_WITH_* markers there so a git pull cannot clobber them, chain behind
errorlevel guards, wire to Task Scheduler.

CROWDMON_STORE is a separate store ROOT, so cotdatas push-to-server.cmd does not
carry the panel and no exclusion-list change would make it. Hence a second push
script rather than a flag.

push-panel.cmd pushes dated directories first and manifest.json last, and here that
ordering is load-bearing rather than tidy: the reader resolves the current week
THROUGH the manifest, so a manifest naming a directory that has not landed is caught
and reported as a partial sync, while the reverse silently serves last week as
though it were this week. Same reason there is no latest symlink. --delete prunes a
week the publisher has dropped from its rolling window, and unlike the cotdata push
there is no consumer-owned directory on the far side to exclude.

No service restart after a push: the readers cache is keyed on the manifests mtime,
unlike a cotdata store push whose consumers hold an in-process indexer.

Also fixes a claim from #67. That PR said the publish-location correction was
stated in all three places; the merge touched four files and bin/publish_damage.sh
was not among them, so it was two. The block is in the script now, pointing at the
Windows templates rather than at an inline invocation.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant