Skip to content

refactor(vintage): drop the flow decomposition, which was a less capable copy - #93

Merged
mspinola merged 1 commit into
mainfrom
claude/drop-flow-decompose
Aug 2, 2026
Merged

mspinola merged 1 commit into
mainfrom
claude/drop-flow-decompose

Conversation

@mspinola

@mspinola mspinola commented Aug 2, 2026

Copy link
Copy Markdown
Owner

Removes vintage_flow.decompose and the cotdata-vintage flow subcommand. Requested after the duplication was measured from the consumer side.

The two were not alternatives, they were one function

vintage_flow.decompose (built 2026-07-30) and crowdmon.futures.flow.decompose (built 2026-08-01) were carried side by side as "defensible alternatives answering slightly different questions". Measured:

cotdata.vintage_flow.decompose  ==  crowdmon.futures.flow.decompose
                                    at tolerance=1.0, gap rule off

On 27 markets and 135,835 transitions from 2006 to 2026:

result
row sets identical, 135,835 both, 0 left-only, 0 right-only
label agreement 100.000000%, zero mismatches
d_long, d_short, d_net identical on every row

Not a coincidence. Both take the dominant leg as argmax(|ΔLong|, |ΔShort|), both break exact ties to the long leg, and both treat a doubly-unmoved week as quiet unconditionally. Setting tolerance=1.0 makes nothing ever mixed, which is precisely this classifier.

So this copy was never a second opinion, it was the less capable one: it could not decline to label a genuinely two-sided week, and it differenced across a 294-day absence as though it were a week. The general implementation stays in crowdmon, where the module spec puts the positioning engine.

The dedup could not go the other way. crowdmon/tests/test_boundaries.py forbids cotdata from importing crowdmon, because a producer depending on its consumer inverts the dependency direction. The equivalence is asserted from that side, in crowdmon/tests/test_flow_equivalence.py, and written up as crowdmon/docs/design/amendments-2026-08-02.md §B29.

No deprecation path, and the reason is checkable

claim evidence
not exported absent from cotdata.__all__
never documented no mention in CHANGELOG.md under any version
not in a published release PyPI carries 0.1.0, which predates the whole vintage subsystem. The 0.2.0 note says in terms that "tagging it is not publishing it"
one caller the CLI subcommand removed alongside it

Kept, dropped, renamed

Kept. zero_sum_check, from_vintage, from_current_store. The zero-sum identity is a claim about cotdata's own parse, and crowdmon.futures.cot_adapter runs it on every load.

Renamed. cotdata-vintage flow becomes cotdata-vintage zero-sum, keeping the half of that command that was cotdata's own and adding the rounding-tolerance count plus a report of non-weekly intervals (COT was fortnightly before 1992-10-13). A command named flow that no longer decomposes flow would be worse than a renamed one. Verified against the real store, and it reproduces §7 Finding 1 exactly: gold 2026-07-21, OI 383,368, both sides 351,385, gap 31,983, and 1926/1926 weeks balanced.

Dropped deliberately. min_frac_oi, the optional dead zone. It defaulted to 0.0, nothing set it, and the surviving implementation answers the same problem with a dominance tolerance that has a sensitivity sweep attached. docs/design/cot_vintage.md §7 records where that argument turned out half right: parameter-free was a real virtue, and it cost the ability to say a week was two-sided, which is the modal outcome at 60% of weeks.

Coverage did not vanish with the tests

Every behaviour the removed tests pinned is pinned on the consuming side, and one that was not already covered there was ported rather than dropped: oi_corroborates is live code in crowdmon but its only test in the workspace lived here. It moved in crowdmon#27, with a second case the original did not cover (the column must be null on mixed/quiet/gap). tests/test_vintage_flow.py carries the mapping table.

A new test_decompose_is_gone_and_stays_gone guards re-adding it out of habit. That is not hypothetical: this classifier was already re-derived once, by a session that did not know this module had held it since 2026-07-30.

docs/design/cot_vintage.md §7 is kept rather than deleted. What it records is a result about the canonical schema obtained by running a decomposition over it; the zero-sum sweep, the spreading finding and the validate() finding do not depend on where the classifier lives.

Merge order

Either order is safe, verified both ways rather than reasoned about:

against result
crowdmon PR #27 vs this branch 433 passed, the 4 equivalence tests skip with a stated reason
crowdmon PR #27 vs current cotdata main 437 passed, equivalence enforced

Checks

  • 262 tests pass, ruff clean
  • Rebased onto a8a2477 (Release 0.2.0), which landed mid-work. pyproject.toml and __init__.py are untouched and stay at 0.2.0
  • No em dashes in added prose

🤖 Generated with Claude Code

…ble copy

vintage_flow.decompose and crowdmon.futures.flow.decompose were carried side by
side since 2026-08-01 as "defensible alternatives answering slightly different
questions". Measured, they are one function:

  cotdata.vintage_flow.decompose == crowdmon.futures.flow.decompose
                                    at tolerance=1.0, gap rule off

On 27 markets and 135,835 transitions from 2006 to 2026: identical row sets,
100.000000% label agreement, zero mismatches, and d_long/d_short/d_net identical
on every row. Both take the dominant leg as argmax(|dLong|,|dShort|), both break
exact ties to the long leg, and both treat a doubly-unmoved week as quiet
unconditionally, so setting tolerance=1.0 (nothing is ever mixed) reproduces this
classifier exactly.

This copy was therefore not a second opinion but the less capable one. It could
not decline to label a genuinely two-sided week, and it differenced across a
294-day absence as though it were a week. The general implementation stays in
crowdmon, where the module spec puts the positioning engine.

The dedup could not go the other way: crowdmon/tests/test_boundaries.py forbids
cotdata from importing crowdmon, since a producer depending on its consumer
inverts the dependency direction. The equivalence is asserted from that side, in
crowdmon/tests/test_flow_equivalence.py, and written up in
crowdmon/docs/design/amendments-2026-08-02.md B29.

NO DEPRECATION PATH, and the reason is checkable rather than asserted. decompose
was never in cotdata.__all__, has never appeared in CHANGELOG.md under any
version, and is in no published release: PyPI carries 0.1.0, which predates the
whole vintage subsystem, and the 0.2.0 note says in terms that tagging is not
publishing. Its only caller was the CLI subcommand removed with it.

KEPT. zero_sum_check, from_vintage and from_current_store stay. The zero-sum
identity is a claim about cotdata's own parse, and crowdmon.futures.cot_adapter
runs it on every load. The `flow` subcommand becomes `zero-sum`, keeping the half
that was cotdata's own and adding the rounding-tolerance count and a report of
non-weekly intervals (COT was fortnightly before 1992-10-13); a command named
`flow` that no longer decomposes flow would be worse than a renamed one.

DROPPED DELIBERATELY. min_frac_oi, the optional dead zone, has no equivalent in
the surviving implementation. It defaulted to 0.0, nothing set it, and the
dominance tolerance answers the same problem with a sensitivity sweep attached.
docs/design/cot_vintage.md section 7 records where that argument turned out to be
half right: parameter-free was a real virtue and it cost the ability to say a
week was two-sided, which is the modal outcome at 60% of weeks.

Section 7 is kept rather than deleted. What it records is a result about the
canonical SCHEMA, obtained by running a decomposition over it, and the zero-sum
sweep, the spreading finding and the validate() finding do not depend on where
the classifier lives.

262 tests pass, ruff clean. crowdmon's own suite passes against this branch (433
passed, its four equivalence tests skipping with a stated reason) and against
current main (437 passed, equivalence enforced), so the two can merge in either
order without a red build.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@mspinola
mspinola merged commit 95e9f15 into main Aug 2, 2026
5 checks passed
@mspinola
mspinola deleted the claude/drop-flow-decompose branch August 2, 2026 20:59
mspinola added a commit that referenced this pull request Aug 2, 2026
0.2.0 was tagged earlier today and never uploaded, because this repo had no release
workflow at the time and the alternative was a hand-rolled twine upload with a stored
token. That tag stands as a git tag and nothing more. Main has since moved past it
(#93 removed the flow decomposition), so publishing the tagged tree would ship
something that was never main's head, and re-pointing the tag would change what
0.2.0 means after the fact.

0.3.0 is what main actually is. Minor rather than patch because a public CLI
subcommand is gone, and everything from 0.2.0 rides along, so a consumer stepping up
from PyPI's 0.1.0 gets the whole vintage subsystem in one move.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
mspinola added a commit that referenced this pull request Aug 2, 2026
* ci: tag-triggered PyPI publish via Trusted Publishing

cotdata had no release workflow, so publishing meant a manual twine upload with a
stored API token from somebody's laptop. crucible already solved this the other way,
with OIDC Trusted Publishing and no stored credentials, and one publish mechanism
across the fleet is one thing to get right and one thing to audit. This is that
workflow, adapted.

Two guards rather than crucible's one, because cotdata carries its version in two
places: pyproject.toml and src/cotdata/__init__.py. A release that bumps one and
forgets the other installs fine and then lies about itself at runtime, which is
exactly the kind of thing nobody checks by hand twice.

Does not publish anything on its own. It needs a one-time GitHub publisher on PyPI
(owner mspinola, repo cotdata, workflow release.yml, environment pypi) which can
only be configured from the project owner's PyPI account, and it fires on a tag push
whose version matches pyproject.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* Release 0.3.0: the first PyPI publish since 0.1.0

0.2.0 was tagged earlier today and never uploaded, because this repo had no release
workflow at the time and the alternative was a hand-rolled twine upload with a stored
token. That tag stands as a git tag and nothing more. Main has since moved past it
(#93 removed the flow decomposition), so publishing the tagged tree would ship
something that was never main's head, and re-pointing the tag would change what
0.2.0 means after the fact.

0.3.0 is what main actually is. Minor rather than patch because a public CLI
subcommand is gone, and everything from 0.2.0 rides along, so a consumer stepping up
from PyPI's 0.1.0 gets the whole vintage subsystem in one move.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant