Repository navigation
Project the trendline with fixed weights, not a polyfit per row (0.14.4) - #49
Merged
Merged
Conversation
is_down_trend_line_break fitted a least-squares line to every trailing 5-bar window with np.polyfit through rolling.apply: 93,546 fits for the 42-market universe on both bases, the largest single cost in append_trading_signals (~36% of it under the profiler). For a fixed window the line through (0..n-1, y) projected to x=n is a fixed linear combination of the y values, w_i = 1/n + (x_i - xbar)(n - xbar)/Sxx, so the rolling fit is one sliding-window dot product. Same NaN rule as rolling(window).apply. Values agree with polyfit to 7.4e-13 relative (lstsq sums in a different order), so not bit for bit. The break compares a close against the projection, so exact ties are decided by rounding in both forms: 9 near-ties (4 inside a downtrend) exist in the real store, and all 84 append_trading_signals outputs (42 markets x raw and OI-normalized) are identical to main, every column, dtype and order. New tests pin the vectorised form against the rolling polyfit, NaN for NaN, windows 2-10. append_trading_signals, 84 real calls: 2.73s -> 1.67s. Cold Signal Matrix build (cot-analyzer's first weekly-page render): 1.7s -> 1.1s locally. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
The first render of cot-analyzer's newest weekly report is dominated by the first
get_symbols_databuild per market, and most of that isappend_trading_signals(see mspinola/cot-analyzer#137). Profiling 84 real calls (42 markets x raw and OI-normalized) found one hot spot:is_down_trend_line_breakrannp.polyfitinrolling.apply93,546 times, about 36% of the function.Change
New
conditions.rolling_trendline_projection(series, window). For a fixed window, the least-squares line through (0..n-1, y) projected to x=n is a fixed weighted sum of the y values,w_i = 1/n + (x_i - xbar)(n - xbar)/Sxx. The rolling fit therefore becomes one sliding-window dot product. It keeps the same NaN rule asrolling(window).apply.project_trendlineis kept as the reference, and tests pin the new form against it. Version bumps to 0.14.4.Equivalence
append_trading_signalsoutputs are identical to main in every column, dtype and column order, including those 4.Speed (local, real stores)
append_trading_signals, 84 callsget_matrix_data(None, "Custom")(3 runs)What's left is about 450 small pandas operations per call on ~440-row frames, roughly 30us each, with no single hot spot. The next biggest pieces are the ~66 column inserts per call into a 170-column frame (~2.4ms) and 11 range indexes (~1.8ms). Going further would mean rewriting the signal conditions in numpy, which is a much larger and riskier change and is not attempted here.
Tests
pytestgives 498 passed, 2 skipped (the EFA/EEM proxy tests, since the dummy store has no equity bars).ruff check src testsis clean (0.15.22). cot-analyzer's suite passes against this tree too (886).🤖 Generated with Claude Code