Paper revision 2.1: the control test was never exercised

README.md

@@ -15,7 +15,7 @@
 
 Aetherius watches a list of companies. It reads new headlines and filings, decides conservatively whether each one really mentions a watched company, scores it with fixed, explainable weights, and puts only evidence-backed, adverse items in front of a human reviewer.
 
-**Headline result.** Replayed on 161 real archived headlines from three crises (SVB 2023, Wirecard 2020, FTX 2022), it flagged **9 of 9** affected names before their formal resolution and raised **0 flags on 3 unrelated control names**.
+**Headline result.** Replayed on 161 real archived headlines from three crises (SVB 2023, Wirecard 2020, FTX 2022), it flagged **9 of 9** affected names before their formal resolution. The backtest says nothing about false alarms: its control names never appear in the data ([§4.6](docs/working_paper/detection_timing_backtest_2026-07.md#46-the-control-test-was-not-exercised)).
 
 **The honest qualifier.** In **0 of 9** cases did the first flag come *before* the first headline reporting the stock already falling. The system is fast, auditable **triage** of a market reaction, not a predictor ([working paper §4.5](docs/working_paper/detection_timing_backtest_2026-07.md#45-timing-relative-to-the-market-reaction)).
 
@@ -38,19 +38,20 @@ Aetherius watches a list of companies. It reads new headlines and filings, decid
 
 ## Results
 
-| Event | Mechanism | Affected names flagged before resolution | Median lead to resolution | Controls flagged | First flag before market reaction |
-|---|---|---|---|---|---|
-| SVB-2023 | Regional-bank contagion | 5 / 5 | 2.34 days | 0 / 1 | 0 / 5 |
-| Wirecard-2020 | Accounting fraud | 1 / 1 | 6.60 days | 0 / 1 | 0 / 1 |
-| FTX-2022 | Crypto counterparty contagion | 3 / 3 | 7.12 days | 0 / 1 | 0 / 3 |
-| **All** | | **9 / 9** (95% CI 66–100%) | 2.48 days | **0 / 3** (FP rate 95% CI 0–71%) | **0 / 9** |
+| Event | Mechanism | Affected names flagged before resolution | Median lead to resolution | First flag before market reaction |
+|---|---|---|---|---|
+| SVB-2023 | Regional-bank contagion | 5 / 5 | 2.34 days | 0 / 5 |
+| Wirecard-2020 | Accounting fraud | 1 / 1 | 6.60 days | 0 / 1 |
+| FTX-2022 | Crypto counterparty contagion | 3 / 3 | 7.12 days | 0 / 3 |
+| **All** | | **9 / 9** (95% CI 66–100%) | 2.48 days | **0 / 9** |
 
 "Resolution" means the formal event: receivership, insolvency filing or Chapter 11. The "market reaction" is the first headline in the corpus reporting the stock already falling ([`market_reaction_check.py`](simulations/backtest/market_reaction_check.py)).
 
 ## Limitations
 
 - **Small sample, chosen with hindsight.** There are three famous failures and nine affected names, and both the events and the adverse-term list were chosen knowing the outcomes. The confidence intervals above are wide.
 - **Triage, not anticipation.** Flags arrive together with the press reporting a sell-off.
+- **False alarms are untested.** Each event has a control company, but none appears in its headlines, so "no control was flagged" proves nothing. The live forward test measures false alarms instead.
 - **Known false positives.** Tickers that are English words match ordinary text ("coin prices fall" matches `COIN`). Adverse terms match as substrings ("fall" fires inside "shortfall").
 - **EDGAR adds provenance, not signal.** Filings are scored on metadata (form and date), never on the filing text.
 - **Whitelisted sources only.** Coverage is a fixed list of financial-news domains, which is thin for non-U.S. events.

aetherius/app/services/reporting/target_stress_test.py

@@ -371,8 +371,9 @@ def _executive_summary(
     bullets.append(
         "Detection track record on structurally comparable historical events: "
         "9 of 9 affected names detected across SVB-2023, Wirecard-2020, and FTX-2022; "
-        "0 false positives on 3 unrelated controls. Flags arrived with the market's "
-        "reaction, not before it: lead time is measured to formal resolution, not to the price move."
+        "Flags arrived with the market's reaction, not before it: lead time is measured to "
+        "formal resolution, not to the price move. False alarms were not tested: the control "
+        "names never appear in the backtest corpora."
     )
     return bullets
 

aetherius/app/templates/pdf/target_stress_test.html

@@ -236,7 +236,7 @@ <h1>Target Stress-Test Deck</h1>
     <div class="cover-footnote">
       Detection track record on 3 frozen historical crises &mdash;
       {{ track_record.aggregate.detected_total }} of {{ track_record.aggregate.affected_total }} affected names detected
-      at elevated / high severity, {{ track_record.aggregate.false_positives_total }} false positives on controls.
+      at elevated / high severity. False alarms were not tested (control names never appear in the corpora).
       Flags coincided with, and did not precede, the first press report of each stock falling.
       Full methodology in the working paper referenced on the final page.
       Aetherius is an information and risk-monitoring service, not investment advice.
@@ -409,7 +409,6 @@ <h2 class="section">Historical evidence on structurally comparable events</h2>
           <th style="width: 15%;">Affected</th>
           <th style="width: 15%;">Detected</th>
           <th style="width: 20%;">Median lead to resolution</th>
-          <th style="width: 15%;">FP on control</th>
         </tr>
       </thead>
       <tbody>
@@ -419,15 +418,13 @@ <h2 class="section">Historical evidence on structurally comparable events</h2>
           <td>{{ ev.affected }}</td>
           <td><strong>{{ ev.detected }}</strong></td>
           <td>{{ "%.2f"|format(ev.median_lead_days) }} days</td>
-          <td>{{ ev.false_positives_on_control }}</td>
         </tr>
         {% endfor %}
         <tr style="font-weight: 600; background: #fbf7ec;">
           <td>Aggregate</td>
           <td>{{ track_record.aggregate.affected_total }}</td>
           <td>{{ track_record.aggregate.detected_total }}</td>
           <td>&mdash;</td>
-          <td>{{ track_record.aggregate.false_positives_total }}</td>
         </tr>
       </tbody>
     </table>

aetherius/tests/test_market_reaction_check.py

@@ -33,3 +33,13 @@ def test_no_first_flag_precedes_the_market_reaction() -> None:
     assert len(rows) == 9
     assert sum(1 for r in rows if r["first_flag_reports_price_move"]) == 8
     assert sum(1 for r in rows if (r["flag_lead_over_reaction_hours"] or 0) > 0) == 0
+
+
+def test_controls_never_appear_in_the_frozen_corpora() -> None:
+    """Pins the paper's statement that the control test was not exercised.
+
+    If a fixture is rebuilt so a control name does appear, this fails: update
+    the paper's Section 4.6 and the README before changing this test.
+    """
+    for event in mrc.EVENTS:
+        assert set(mrc.control_coverage(event).values()) == {0}