Repository navigation
Paper revision 2.1: the control test was never exercised - #5
Merged
Merged
Conversation
A pre-publication check found that no observation in any of the three frozen corpora maps to a control name (MSFT, DTE): GDELT returned body matches, but the pipeline scores headlines only and none names the control. The '0 false positives on 3 controls' result was therefore vacuous, and earlier text describing control headlines was wrong. - Paper: new section 4.6, corrected abstract, results table, sections 2.4 and 5.2; adds related work and references, keywords, JEL codes and affiliation; discloses that the 4.5 price-reaction pattern was written after reading the headlines; anchor-removal robustness now stated for all three events (verified). - market_reaction_check.py: control_coverage(); corrected comment. - run_backtest.py prints control coverage next to the false-positive count (metrics files unchanged). - Test pins zero control coverage so the paper must change if fixtures do. - README, site, social card, demo GIF and deck no longer claim 0/3. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01HTqzkhjyuhZZun8Az1tW6C
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
What changed?
A pre-publication check found that no observation in any of the three frozen corpora maps to a control name (MSFT in SVB and FTX, DTE in Wirecard). The fixture builder did query GDELT for each control, and GDELT returned articles that matched in their body text. But the pipeline scores headlines only, and none of those headlines names the control. So "0 false positives on 3 controls" was vacuous. Text in Revisions 1 and 2 describing control headlines ("MSFT appeared in benign product coverage", "DTE appeared … in mundane telecom coverage") was false.
market_reaction_check.py: newcontrol_coverage(), printed with the results; corrected the code comment that claimed the pattern was fixed in advance.run_backtest.py: printscontrols matched in corpus: Nnext to the false-positive count. The metrics files are unchanged.test_controls_never_appear_in_the_frozen_corporapins zero coverage, so the paper must be updated if a fixture is ever rebuilt.Why was this needed? The paper is about to be posted on SSRN under the author's name. A false claim about specificity would undermine the paper's main credibility argument, which is honest reporting.
Validation
pytest -q aetherius/tests): 122 passed.simulations/validation/is unchanged.ruff check aetherius simulations scripts).Scope
aetherius/app/services/...)simulations/backtest/...): diagnostics output onlySafety Checklist
.env, API keys, tokens)simulations/artifacts/, runtime logs)Notes for Reviewers
🤖 Generated with Claude Code
https://claude.ai/code/session_01HTqzkhjyuhZZun8Az1tW6C
Generated by Claude Code