diff --git a/docs/adr/0053-issue-layout-and-entry-facts.md b/docs/adr/0053-issue-layout-and-entry-facts.md index a8b7b6f..0007f6b 100644 --- a/docs/adr/0053-issue-layout-and-entry-facts.md +++ b/docs/adr/0053-issue-layout-and-entry-facts.md @@ -78,7 +78,9 @@ Amended 2026-11-10, after measuring the export of that day (agent's proposal): - *Coverage:* abbreviations ("UC", "PsA") never match MeSH, so a release that writes the disease only that way gives no indication. Company pipeline pages (decision 5) remain the route to nearly every entry. - **The reading job (`catalyst indicate-catalysts`, 2026-11-12):** for each published catalyst with one drug tested alone and no indication, plain code takes the source claim that states the event and its stored copy, and both readers run as above. A found indication is kept as a claim: its quote is the sentence, its value the MeSH identifier, the MeSH name, and the sentence's own words; its field is `indication:` and the catalyst's id, so two catalysts never share a claim row. The second reader's pass is a claim check beside it. Every try, found or refused, is a row of `indication_attempt`, so a re-run never pays to ask again about the same catalyst, claim, prompts, and models. A copy that the strict and browser-like readers read differently is refused without a model call (ADR 0044). The job asks nothing unless `indication_gate_problems` is empty, and a role at its cap stops it. It never writes a catalyst; the writer's pass does, in the next pull request. The writer reads indications only through `indication_attempt`, never by claim field, and uses only each catalyst's latest attempt (by `attempted_at`): a later refusal, an earlier run's disease redrawn included, overrides an earlier find (Opus review of PR 127). A copy refused because the two page readers differ is refused without a model call; it comes back only when a prompt or model version changes. - **Its own gate:** a catalyst whose indication fails these checks publishes as it does today, without an indication. Only the indication waits in the review queue with its reason. No existing check is loosened. This touches the publish gate: test first, and Opus review. -- **Three pull requests:** the MeSH file and matcher (merged, PR 123); the two readers and their checks (PR 124); then the writer, with Opus review. +- **The writer's pass (`catalyst.publish.indication`, 2026-11-12):** plain code under the writer's lock. For each catalyst with no indication, it takes the latest `indication_attempt`; only when that try found one, it checks that the claim is the catalyst's own (`indication:`), that the second reader's pass is stored, and that the quote is still at its place in the stored copy for the current text version (`gate.match_quote`). Then it sets `indication_id` to the MeSH disease's `indication` row (one per disease, no line or biomarker yet) and writes a change row citing the claim, so the read API shows the sentence among the catalyst's sources. An indication already set is never changed. What is printed is the claim's own words (`IndicationView.name`); the MeSH name stays in the `indication` row for search. +- **When it runs:** the weekday extract run asks both readers about up to its document limit of newly published catalysts right after the writer, then runs the indication pass; `catalyst publish-claims` runs the pass too. No workflow, role, or cap changes: the readers spend under the extractor's and verifier's existing daily caps. +- **Pull requests:** the MeSH file and matcher (PR 123); the two readers and their checks (PR 124); the floors (PR 126); the reading job (PR 127); the writer's pass (this one), each with a separate Opus review. ### Order diff --git a/docs/roadmap.md b/docs/roadmap.md index 4ce46df..b1bfcfe 100644 --- a/docs/roadmap.md +++ b/docs/roadmap.md @@ -72,8 +72,8 @@ Claude Code: tick items (`[x]`) in the same pull request that finishes them. - [ ] Facts on every entry (owner request 2026-11-10: indication, phase including preclinical, cohort size, named conferences' dates, pipeline and program links; the methods below and the wait for the first issue are the agent's proposals; ADR 0053, decisions 2 to 6), each its own pull request: - [ ] Indication (ADR 0053, decision 2 as amended): its own quote naming the drug and the disease, matched to one MeSH disease by plain code; gated on its own, so the catalyst still publishes without it; publish gate, test first, Opus review - [x] MeSH file and matcher: `catalyst collect --sources mesh` stores NLM's yearly descriptor file with its hash; `catalyst.mesh` matches whole words to exactly one disease, refusing narrower, negated, broad, and short-form matches (41% of 371 extracted indications; two samples of 40 all correct) - - [x] The indication's own sentence, read by one model and checked by a second (owner decision 2026-11-10; ADR 0053, decision 2 as amended): plain code offers the visible sentences naming the drug; the first reader (extractor role) picks one and copies the disease; plain code checks it is written there and names one MeSH disease; the second reader (verifier role) confirms the sentence states the drug is aimed at it for this event; scored on 20 gold events and 37 planted cases; three earlier routes not merged (their results are in the ADR) - - [ ] The writer: the indication and its change row, with its own review reason + - [x] The indication's own sentence, read by one model and checked by a second (owner decision 2026-11-10; ADR 0053, decision 2 as amended): plain code offers the visible sentences naming the drug; the first reader (extractor role) picks one and copies the disease; plain code checks it is written there and names one MeSH disease; the second reader (verifier role) confirms the sentence states the drug is aimed at it for this event; scored on 20 gold events and 39 planted cases; three earlier routes not merged (their results are in the ADR) + - [x] The writer: the indication and its change row, with its own review reason (ADR 0053, decision 2): floors as owner paths (PR 126); `catalyst indicate-catalysts` keeps every try in `indication_attempt` with its reason (PR 127); the writer's indication pass sets `indication_id` from each catalyst's latest confirmed try after checking its claim, the second reader's pass, and its quote, with a change row citing the claim; the read API prints the sentence's own words; the weekday extract run asks and writes after publishing. Refusal reasons are in `indication_attempt` and the run's summary; the review-queue page shows them when it is built - [ ] Conferences: `config/conferences.toml` lists which meetings to follow and their official pages; each page is stored with its hash and its dates and city are quoted claims through the publish gate; plain code adds them when a quote names a followed meeting; an unnamed one stays as written - [ ] Development stage, preclinical included: a new prompt version with a whole-word check, cross-checked against the registry phase, gold set scored again; adds 🟧 Preclinical to the key - [ ] Pipeline and program pages: found on each company's own domain, stored with their hash, linked on a "Company pages:" line apart from the sources (they back no value), checked weekly @@ -205,6 +205,7 @@ Newest first. Every ADR in `docs/adr` adds a line here. | Date | Decision | | --- | --- | +| 2026-11-12 | ADR 0053, decision 2: the writer's indication pass sets a catalyst's `indication_id` only from its latest `indication_attempt` when that try found one, after checking the claim is the catalyst's own, the second reader's pass is stored, and the quote still matches its stored copy; a change row cites the claim; the read API prints the sentence's words, not the MeSH name; the weekday extract run asks and writes right after publishing, inside the existing caps | | 2026-11-12 | ADR 0053, decision 2: the indication reading job (`catalyst indicate-catalysts`) asks both readers about published catalysts with one drug tested alone and no indication, only while the indication gate is clear; a found indication is a claim (field `indication:`, quote its sentence) with the second reader's pass as a check; every try is kept in `indication_attempt` so none is paid for twice; it never writes a catalyst; plain code also refuses a disease written straight before another MeSH disease term | | 2026-11-12 | ADR 0053, decision 2: floors for the indication readers, at the third run's scores (found_right ≥ 0.133, none_right ≥ 1.0, wrong ≤ 0.0; checker accepts_true ≥ 1.0, refuses_false ≥ 0.962), as owner paths in `scripts/merge_gate.py`; the readers may not run on live data unless their scores match the current prompts, role settings, and label sets and meet the floors | | 2026-11-12 | ADR 0054 (agent's proposal; amends ADR 0010 and ADR 0043): the Gate 1 run of 2026-10-26 timed out, not failed: the stored-copy check reads copies one at a time, and the link check reached 2,031 of the stored ClinicalTrials.gov links (all working) in 45 minutes; `snapshots-verify --workers 8` with a 30-minute limit, and Gate 1's link step checks the links published records cite with ADR 0010's HTTP 200 rule (`check-links --published --strict`), narrowing ADR 0010's scope from every stored link; step limits add up to 145 of the job's 150 minutes | diff --git a/pipeline/catalyst/api/read.py b/pipeline/catalyst/api/read.py index 8ce47bc..1137ffd 100644 --- a/pipeline/catalyst/api/read.py +++ b/pipeline/catalyst/api/read.py @@ -15,7 +15,7 @@ from contextlib import contextmanager from dataclasses import dataclass, replace from datetime import date, timedelta -from typing import Any +from typing import Any, cast from uuid import UUID from pydantic import JsonValue @@ -37,6 +37,7 @@ WindowView, ) from catalyst.db.tables import TABLES +from catalyst.extract.agent import has_words from catalyst.logs import log_event from catalyst.schemas import ( Catalyst, @@ -73,6 +74,7 @@ OPEN_STATUSES = (CatalystStatus.UPCOMING, CatalystStatus.SLIPPED) BACKED_FIELDS = tuple(f.value for f in CatalystField if f is not CatalystField.DRUGS) DRUGS = CatalystField.DRUGS.value +INDICATION = CatalystField.INDICATION_ID.value TRIALS = "trials" # a source's `fields` entry for the quote that names a trial (ADR 0039) DATE_FIELDS = frozenset(f.value for f in CatalystField if f.value.startswith("window_")) MAX_LIMIT = 200 @@ -283,7 +285,7 @@ def _details(session: Session, rows: Sequence[Row[Any]]) -> list[CatalystDetail] ids = [row.id for row in rows] changes: dict[UUID, list[Row[Any]]] = defaultdict(list) for change in session.execute( - select(_CHANGE, *_SOURCE_COLUMNS) + select(_CHANGE, *_SOURCE_COLUMNS, _CLAIM.c.value.label("claim_value")) .join(_CLAIM, _CLAIM.c.id == _CHANGE.c.claim_id) .join(_DOC, _DOC.c.id == _CLAIM.c.source_document_id) .where(_CHANGE.c.catalyst_id.in_(ids)) @@ -379,7 +381,7 @@ def _build( catalyst_type=row.catalyst_type, title=row.title, company=companies[row.company_id], - indication=indications.get(row.indication_id) if row.indication_id else None, + indication=_shown_indication(indications.get(row.indication_id), latest.get(INDICATION)), drugs=[DrugView(id=link.drug_id, name=link.preferred_name, role=link.role) for link in links], trials=[trial for _, trial in trials], window=WindowView( @@ -396,6 +398,22 @@ def _build( ) +def _shown_indication(indication: IndicationView | None, change: Row[Any] | None) -> IndicationView | None: + """Return the indication named in its own source's words, not the MeSH name (ADR 0053, decision 2). + + The words are those the claim behind the latest `indication_id` change kept from its sentence; + the MeSH name is shown only when that claim kept none. + """ + if indication is None or change is None: + return indication + stored: object = change.claim_value + value = cast(dict[str, object], stored) if isinstance(stored, dict) else {} + words = value.get("words") + if isinstance(words, str) and words.strip() and has_words(change.quote, words): + return indication.model_copy(update={"name": words}) + return indication # words the cited quote does not hold are never shown + + def drugs_value(links: Sequence[Row[Any]]) -> list[JsonValue]: """Return the links as a `drugs` change states them: `{drug_id, role}` pairs, sorted (ADR 0025).""" pairs = sorted({(str(link.drug_id), str(link.role)) for link in links}) diff --git a/pipeline/catalyst/cli.py b/pipeline/catalyst/cli.py index 5e60381..b238984 100644 --- a/pipeline/catalyst/cli.py +++ b/pipeline/catalyst/cli.py @@ -68,6 +68,7 @@ render_html, render_page, ) +from catalyst.publish.indication import publish_indications from catalyst.publish.run import decide_claims from catalyst.publish.sanity import check_safe_to_write from catalyst.publish.write import publish_claims @@ -178,7 +179,9 @@ def _parser() -> argparse.ArgumentParser: "extract-catalysts", help="Ask the extractor for the catalysts in stored press releases (costs money).", ) - extract.add_argument("--limit", type=int, default=50, help="Most documents to read in this run.") + extract.add_argument( + "--limit", type=_at_least_one, default=50, help="Most documents to read in this run." + ) extract.add_argument( "--since", type=date.fromisoformat, default=None, help="Earliest publication date, YYYY-MM-DD." ) @@ -599,6 +602,7 @@ def _extract(args: argparse.Namespace) -> int: second = {"error": f"{type(error).__name__}: the second reading did not finish"} decisions = _decide_after(sessionmaker(engine), store, checks) published = _publish_after(sessionmaker(engine), store, decisions, config.roles[VERIFIER_ROLE].model) + indications = _indications_after(sessionmaker(engine), store, client, published, args.limit) finally: engine.dispose() if args.out is not None: @@ -610,10 +614,11 @@ def _extract(args: argparse.Namespace) -> int: "second_reading": second, "decisions": decisions, "published": published, + "indications": indications, } print(json.dumps(summary, indent=2)) second_failed = "error" in second or second.get("documents_failed") or second.get("stopped_early") - gate_failed = "error" in decisions or "error" in published + gate_failed = "error" in decisions or "error" in published or "error" in indications failed = summary["documents_failed"] or summary["stopped_early"] or second_failed or gate_failed return 1 if failed else 0 @@ -677,6 +682,38 @@ def _publish_after( return {"error": f"{type(error).__name__}: publishing did not finish"} +def _indications_after( + session_factory: sessionmaker[Session], + store: SnapshotStore, + client: ModelClient, + published: dict[str, Any], + limit: int, +) -> dict[str, Any]: + """Find newly published catalysts' indications, then write the confirmed ones (ADR 0053). + + After the writer, so new catalysts are asked about the same day; skipped when publishing did + not finish. A blocked gate or a role at its cap is reported, not an error: the catalysts + simply wait for a later run. + """ + if "error" in published or "skipped" in published: + return {"skipped": "publishing did not finish, so no indication was asked for"} + failures = (SQLAlchemyError, SnapshotStoreError, httpx.HTTPError, SyntaxError, ValueError, OSError) + result: dict[str, Any] = {} + try: + result["read"] = indicate_catalysts( + session_factory, store, client, limit=limit, now=_utc_now + ).summary() + except failures as error: + result["read"] = {"error": f"{type(error).__name__}: the indication readers did not finish"} + try: # its own step, so finds stored by earlier runs are written even when the readers fail + result |= publish_indications(session_factory, store, _utc_now).summary() + except failures as error: + result["write"] = {"error": f"{type(error).__name__}: the indication pass did not finish"} + if "error" in result["read"] or "write" in result: + result["error"] = "the indication step did not finish" + return result + + def _resolve(_args: argparse.Namespace) -> int: try: database = settings.database_url() @@ -776,9 +813,10 @@ def _publish_claims(args: argparse.Namespace) -> int: try: model = load_config().roles[VERIFIER_ROLE].model report = publish_claims(sessionmaker(engine), store, _utc_now, verifier_model=model) + indications = publish_indications(sessionmaker(engine), store, _utc_now) finally: engine.dispose() - print(json.dumps(report.summary(), indent=2)) + print(json.dumps(report.summary() | indications.summary(), indent=2)) return 0 diff --git a/pipeline/catalyst/extract/indicate.py b/pipeline/catalyst/extract/indicate.py index b2d99ad..8d4d0d4 100644 --- a/pipeline/catalyst/extract/indicate.py +++ b/pipeline/catalyst/extract/indicate.py @@ -301,6 +301,19 @@ def _tasks(session: Session, report: IndicationReport, models: tuple[str, str]) return tasks +def _load_index(session_factory: sessionmaker[Session], store: SnapshotStore) -> MeshIndex | str: + """Return the stored MeSH index, or why the job cannot read it, so the run stops asking.""" + try: + with session_factory() as session: + index = stored_index(session, store) + except (SnapshotStoreError, httpx.HTTPError, SyntaxError, ValueError) as error: + # A storage error, a changed copy, or a file that is not MeSH XML (review of PR 128). + return f"the stored MeSH file could not be read ({type(error).__name__})" + if index is None: + return "no MeSH descriptor file is stored yet; run `catalyst collect --sources mesh`" + return index + + def indicate_catalysts( session_factory: sessionmaker[Session], store: SnapshotStore, @@ -315,16 +328,16 @@ def indicate_catalysts( report = IndicationReport(blocked=gate()) if report.blocked: return report # no model is called without scores that meet their floors - if index is None: - with session_factory() as session: - index = stored_index(session, store) - if index is None: - report.blocked = ["no MeSH descriptor file is stored yet; run `catalyst collect --sources mesh`"] - return report models = (client.model_for(ROLE), client.model_for(VERIFIER_ROLE)) provider = load_config().roles[ROLE].provider with read_session(session_factory) as session: tasks = _tasks(session, report, models) + if not tasks: + return report # nothing to ask about, so the MeSH file is not read + loaded = index if index is not None else _load_index(session_factory, store) + if isinstance(loaded, str): + report.blocked = [loaded] + return report for task in tasks[:limit]: report.considered += 1 try: @@ -334,7 +347,7 @@ def indicate_catalysts( if page.differs: outcome = Outcome(IndicationMiss.COPY_DIFFERS) # such a copy's claims go to a person else: - outcome = find_indication(client, text, page.visible, task.drug, task.quote, index) + outcome = find_indication(client, text, page.visible, task.drug, task.quote, loaded) except SpendCapReachedError as error: report.considered -= 1 report.stopped = str(error) diff --git a/pipeline/catalyst/publish/indication.py b/pipeline/catalyst/publish/indication.py new file mode 100644 index 0000000..5980f49 --- /dev/null +++ b/pipeline/catalyst/publish/indication.py @@ -0,0 +1,188 @@ +"""The writer's indication pass: a catalyst's disease, from its latest confirmed try (ADR 0053, decision 2). + +Plain code, no model calls, under the writer's lock (ADR 0018, part 5b). For each published +catalyst with no indication, it takes the catalyst's latest `indication_attempt`; only when that +try found one, it checks again before writing: + +- the claim is this catalyst's own indication claim (`indication:`); +- the second reader's pass (`indication_confirmed`) is stored beside it; +- the quote is, character for character, at its stored place in the stored copy, for the current + text version (`gate.match_quote`), so every published value keeps its source words. + +Then it writes the catalyst's `indication_id` and a change row citing the claim. A later refusal +always wins over an earlier find, because only the latest try counts; claims are never chosen by +field. It never changes an indication already set, deletes, or unpublishes. Safe to re-run. +""" + +from __future__ import annotations + +import logging +from collections import Counter +from collections.abc import Callable +from dataclasses import dataclass, field +from datetime import datetime +from typing import Any +from uuid import UUID + +from pydantic import ValidationError +from sqlalchemy import func, select, update +from sqlalchemy.dialects.postgresql import distinct_on, insert +from sqlalchemy.orm import Session, sessionmaker + +from catalyst.db.tables import TABLES +from catalyst.extract.agent import has_words +from catalyst.extract.indicate import CONFIRMED_CHECK, claim_field +from catalyst.logs import log_event +from catalyst.publish import gate +from catalyst.publish.live import claim_value +from catalyst.publish.write import LOCK +from catalyst.schemas import ( + Catalyst, + CatalystChange, + CatalystField, + CheckOutcome, + Claim, + ClaimCheck, + Indication, + IndicationAttempt, + IndicationResult, + SourceDocument, +) +from catalyst.snapshots import SnapshotStore + +_log = logging.getLogger(__name__) + +NOT_ITS_OWN = "the claim is not this catalyst's indication claim" +NOT_CONFIRMED = "the second reader's pass is not stored beside the claim" +NO_DISEASE = "the claim names no MeSH disease" +WORDS_NOT_IN_QUOTE = "the words to print are not in the claim's quote" + + +@dataclass +class IndicationWrites: + """What one pass wrote, and why each found indication it did not write waits.""" + + written: int = 0 + held: Counter[str] = field(default_factory=Counter[str]) + + def summary(self) -> dict[str, Any]: + """Return the counts as plain JSON.""" + return {"indications_written": self.written, "indications_held": dict(sorted(self.held.items()))} + + +def _latest_found(session: Session) -> list[Any]: + """Return each catalyst's latest try when it found an indication, for catalysts that have none.""" + attempt, catalyst = TABLES[IndicationAttempt], TABLES[Catalyst] + latest = ( + select(attempt) + .ext(distinct_on(attempt.c.catalyst_id)) + .order_by(attempt.c.catalyst_id, attempt.c.attempted_at.desc(), attempt.c.id.desc()) + .subquery() + ) + query = ( + select(latest) + .join(catalyst, catalyst.c.id == latest.c.catalyst_id) + .where(latest.c.outcome == IndicationResult.FOUND.value, catalyst.c.indication_id.is_(None)) + .order_by(latest.c.catalyst_id) + ) + return list(session.execute(query)) + + +def _claim_and_copy(session: Session, claim_id: UUID) -> Any: + claim, document = TABLES[Claim], TABLES[SourceDocument] + return session.execute( + select(claim, document.c.snapshot_key, document.c.sha256, document.c.content_type) + .join(document, document.c.id == claim.c.source_document_id) + .where(claim.c.id == claim_id) + ).one() + + +def _confirmed(session: Session, claim_id: UUID) -> bool: + check = TABLES[ClaimCheck] + outcome = session.execute( + select(check.c.outcome).where(check.c.claim_id == claim_id, check.c.check_name == CONFIRMED_CHECK) + ).scalar_one_or_none() + return outcome == CheckOutcome.PASS.value + + +def _indication_id(session: Session, row: Indication) -> UUID: + """Return the indication row for this MeSH disease with no line or biomarker, adding it once.""" + table = TABLES[Indication] + mesh_id = row.mesh_id + inserted = session.execute( + insert(table).values(**row.model_dump()).on_conflict_do_nothing().returning(table.c.id) + ).scalar_one_or_none() + if inserted is not None: + return inserted + return session.execute( + select(table.c.id).where( + table.c.mesh_id == mesh_id, table.c.line_of_therapy.is_(None), table.c.biomarker.is_(None) + ) + ).scalar_one() + + +def _why_not( + session: Session, store: SnapshotStore, attempt: Any +) -> tuple[str | None, Any, Indication | None]: + """Return why this found indication may not be written (None when it may), its claim row, and its disease. + + Every value to be written is checked again here, so one bad row is held on its own and never + stops the pass for the others (Opus review of PR 128). + """ + row = _claim_and_copy(session, attempt.indication_claim_id) + if row.subject_id != attempt.catalyst_id or row.field != claim_field(attempt.catalyst_id): + return NOT_ITS_OWN, row, None + if not _confirmed(session, row.id): + return NOT_CONFIRMED, row, None + value = claim_value(row.value) + try: + disease = Indication.model_validate({"mesh_id": value.get("mesh_id"), "name": value.get("mesh_name")}) + except ValidationError: + return NO_DISEASE, row, None + words = value.get("words") + if not isinstance(words, str) or not has_words(row.quote, words): + return WORDS_NOT_IN_QUOTE, row, None + claim = Claim.model_validate({k: getattr(row, k) for k in Claim.model_fields}) + quote = gate.match_quote(claim, gate.read_copy(store, row.snapshot_key, row.sha256, row.content_type)) + if quote.outcome is not CheckOutcome.PASS: + return quote.reason or "the quote check failed", row, None + return None, row, disease + + +def write_indications(session: Session, store: SnapshotStore, at: datetime) -> IndicationWrites: + """Write each catalyst's indication from its latest confirmed try, inside the caller's transaction.""" + report = IndicationWrites() + for attempt in _latest_found(session): + reason, row, disease = _why_not(session, store, attempt) + if reason is not None or disease is None: + reason = reason or NO_DISEASE + report.held[reason] += 1 + log_event(_log, "indication_held", catalyst_id=str(attempt.catalyst_id), reason=reason) + continue + indication_id = _indication_id(session, disease) + table = TABLES[Catalyst] + session.execute( + update(table).where(table.c.id == attempt.catalyst_id).values(indication_id=indication_id) + ) + change = CatalystChange( + catalyst_id=attempt.catalyst_id, + field=CatalystField.INDICATION_ID, + new_value=str(indication_id), + claim_id=row.id, + learned_at=at, + ) + session.execute(insert(TABLES[CatalystChange]).values(**change.model_dump())) + report.written += 1 + log_event(_log, "indication_written", catalyst_id=str(attempt.catalyst_id), claim_id=str(row.id)) + return report + + +def publish_indications( + session_factory: sessionmaker[Session], store: SnapshotStore, now: Callable[[], datetime] +) -> IndicationWrites: + """Run the indication pass on its own, under the writer's lock, in one transaction.""" + with session_factory.begin() as session: + session.execute(select(func.pg_advisory_xact_lock(LOCK))) + report = write_indications(session, store, now()) + log_event(_log, "indications_published", **report.summary()) + return report diff --git a/tests/extract/test_indicate.py b/tests/extract/test_indicate.py index d688111..3817b35 100644 --- a/tests/extract/test_indicate.py +++ b/tests/extract/test_indicate.py @@ -14,11 +14,14 @@ from typing import Any from uuid import UUID +import httpx +import pytest from sqlalchemy import insert, select from sqlalchemy.orm import Session, sessionmaker from catalyst.collectors.documents import FetchedDocument, store_fetched from catalyst.db.tables import TABLES +from catalyst.extract import indicate from catalyst.extract.indicate import CONFIRMED_CHECK, REDRAWN, claim_field, indicate_catalysts from catalyst.extract.text import document_text, find_quote from catalyst.llm.client import Completion, ModelClient @@ -48,7 +51,7 @@ SourceDocument, SourceTier, ) -from catalyst.snapshots import LocalSnapshotStore +from catalyst.snapshots import LocalSnapshotStore, SnapshotStoreError FIXTURE = Path(__file__).parents[1] / "fixtures" / "mesh" / "desc2026_excerpt.xml.gz" EVENT = "Topline data from the Phase 3 RAINIER trial of zentocel are expected in the second half of 2027." @@ -66,7 +69,7 @@ def _add(session: Session, record: Entity) -> None: session.execute(insert(TABLES[type(record)]).values(**record.model_dump())) -def _seed( +def seed_catalyst( factory: sessionmaker[Session], store: LocalSnapshotStore, n: int = 1, role: DrugRole = DrugRole.ALONE ) -> UUID: """Publish one catalyst the way the writer does, from a claim naming the drug as the release does.""" @@ -136,7 +139,7 @@ def _seed( @dataclass -class _Fake: +class FakeReaders: """The first reader picks the sentence naming `disease`; the second answers as the test sets.""" disease: str | None = "melanoma" @@ -158,7 +161,7 @@ def complete( return Completion(json.dumps(answer), model, "req", "end_turn", TokenUsage(input=100, output=20)) -def _client(fake: _Fake, cap: str = "5") -> ModelClient: +def _client(fake: FakeReaders, cap: str = "5") -> ModelClient: price = Price(input=Decimal(2), cache_write=Decimal(3), cache_read=Decimal(0), output=Decimal(10)) role = RoleConfig(provider="fake", model="claude-sonnet-5-5", max_tokens=1000, daily_cap_usd=Decimal(cap)) config = ModelConfig( @@ -167,8 +170,8 @@ def _client(fake: _Fake, cap: str = "5") -> ModelClient: return ModelClient(config, {"fake": fake}, InMemoryLedger(), now=lambda: NOW) -def _run( - factory: sessionmaker[Session], store: LocalSnapshotStore, fake: _Fake, cap: str = "5" +def run_readers( + factory: sessionmaker[Session], store: LocalSnapshotStore, fake: FakeReaders, cap: str = "5" ) -> dict[str, Any]: report = indicate_catalysts( factory, store, _client(fake, cap), limit=10, index=_index(), now=lambda: NOW, gate=lambda: [] @@ -176,7 +179,7 @@ def _run( return report.summary() -def _all(factory: sessionmaker[Session], model: type[Entity]) -> list[Any]: +def all_rows(factory: sessionmaker[Session], model: type[Entity]) -> list[Any]: with factory() as session: return list(session.execute(select(TABLES[model])).all()) @@ -185,37 +188,37 @@ def test_a_confirmed_indication_is_kept_as_a_claim_whose_quote_is_its_sentence( session_factory: sessionmaker[Session], tmp_path: Path ) -> None: store = LocalSnapshotStore(tmp_path) - catalyst_id = _seed(session_factory, store) - summary = _run(session_factory, store, _Fake()) + catalyst_id = seed_catalyst(session_factory, store) + summary = run_readers(session_factory, store, FakeReaders()) assert (summary["considered"], summary["found"], summary["refused"]) == (1, 1, {}) - (claim,) = [c for c in _all(session_factory, Claim) if c.field == claim_field(catalyst_id)] + (claim,) = [c for c in all_rows(session_factory, Claim) if c.field == claim_field(catalyst_id)] assert claim.quote == TARGET assert claim.subject_id == catalyst_id assert claim.value["words"] == "melanoma" assert claim.value["mesh_name"] == "Melanoma" assert document_text(RELEASE, "text/html")[claim.quote_start : claim.quote_end] == TARGET - (check,) = [c for c in _all(session_factory, ClaimCheck) if c.claim_id == claim.id] + (check,) = [c for c in all_rows(session_factory, ClaimCheck) if c.claim_id == claim.id] assert (check.check_name, check.outcome) == (CONFIRMED_CHECK, "pass") - (attempt,) = _all(session_factory, IndicationAttempt) + (attempt,) = all_rows(session_factory, IndicationAttempt) assert (attempt.outcome, attempt.indication_claim_id) == ("found", claim.id) def test_the_job_never_writes_a_catalyst(session_factory: sessionmaker[Session], tmp_path: Path) -> None: store = LocalSnapshotStore(tmp_path) - catalyst_id = _seed(session_factory, store) - changes = len(_all(session_factory, CatalystChange)) - _run(session_factory, store, _Fake()) - (row,) = [c for c in _all(session_factory, Catalyst) if c.id == catalyst_id] + catalyst_id = seed_catalyst(session_factory, store) + changes = len(all_rows(session_factory, CatalystChange)) + run_readers(session_factory, store, FakeReaders()) + (row,) = [c for c in all_rows(session_factory, Catalyst) if c.id == catalyst_id] assert row.indication_id is None - assert len(_all(session_factory, CatalystChange)) == changes + assert len(all_rows(session_factory, CatalystChange)) == changes def test_a_re_run_asks_nothing_again(session_factory: sessionmaker[Session], tmp_path: Path) -> None: store = LocalSnapshotStore(tmp_path) - _seed(session_factory, store) - _run(session_factory, store, _Fake()) - again = _Fake() - summary = _run(session_factory, store, again) + seed_catalyst(session_factory, store) + run_readers(session_factory, store, FakeReaders()) + again = FakeReaders() + summary = run_readers(session_factory, store, again) assert again.sent == [] assert (summary["considered"], summary["already_tried"]) == (0, 1) @@ -224,22 +227,22 @@ def test_a_refusal_is_kept_with_its_reason_and_no_claim( session_factory: sessionmaker[Session], tmp_path: Path ) -> None: store = LocalSnapshotStore(tmp_path) - catalyst_id = _seed(session_factory, store) - summary = _run(session_factory, store, _Fake(states=False, reason="other_drug")) + catalyst_id = seed_catalyst(session_factory, store) + summary = run_readers(session_factory, store, FakeReaders(states=False, reason="other_drug")) assert summary["found"] == 0 assert summary["refused"] == {"the second reader did not confirm the sentence states the disease": 1} - (attempt,) = _all(session_factory, IndicationAttempt) + (attempt,) = all_rows(session_factory, IndicationAttempt) assert (attempt.outcome, attempt.indication_claim_id) == ("refused", None) assert attempt.reason == "the second reader did not confirm the sentence states the disease" - assert not [c for c in _all(session_factory, Claim) if c.field == claim_field(catalyst_id)] + assert not [c for c in all_rows(session_factory, Claim) if c.field == claim_field(catalyst_id)] def test_no_model_is_called_while_the_gate_finds_a_problem( session_factory: sessionmaker[Session], tmp_path: Path ) -> None: store = LocalSnapshotStore(tmp_path) - _seed(session_factory, store) - fake = _Fake() + seed_catalyst(session_factory, store) + fake = FakeReaders() report = indicate_catalysts( session_factory, store, @@ -251,15 +254,15 @@ def test_no_model_is_called_while_the_gate_finds_a_problem( ) assert report.blocked == ["indicate_catalysts has no scores.json"] assert fake.sent == [] - assert _all(session_factory, IndicationAttempt) == [] + assert all_rows(session_factory, IndicationAttempt) == [] def test_without_a_stored_mesh_file_nothing_is_asked( session_factory: sessionmaker[Session], tmp_path: Path ) -> None: store = LocalSnapshotStore(tmp_path) - _seed(session_factory, store) - fake = _Fake() + seed_catalyst(session_factory, store) + fake = FakeReaders() report = indicate_catalysts( session_factory, store, _client(fake), limit=10, now=lambda: NOW, gate=lambda: [] ) @@ -272,28 +275,28 @@ def test_a_drug_tested_in_combination_is_left_alone( session_factory: sessionmaker[Session], tmp_path: Path ) -> None: store = LocalSnapshotStore(tmp_path) - _seed(session_factory, store, role=DrugRole.IN_COMBINATION) - fake = _Fake() - summary = _run(session_factory, store, fake) + seed_catalyst(session_factory, store, role=DrugRole.IN_COMBINATION) + fake = FakeReaders() + summary = run_readers(session_factory, store, fake) assert (summary["considered"], fake.sent) == (0, []) def test_a_role_at_its_cap_stops_the_job(session_factory: sessionmaker[Session], tmp_path: Path) -> None: store = LocalSnapshotStore(tmp_path) - _seed(session_factory, store) - summary = _run(session_factory, store, _Fake(), cap="0.000001") + seed_catalyst(session_factory, store) + summary = run_readers(session_factory, store, FakeReaders(), cap="0.000001") assert summary["stopped"] assert summary["considered"] == 0 - assert _all(session_factory, IndicationAttempt) == [] + assert all_rows(session_factory, IndicationAttempt) == [] def test_a_kept_indication_passes_the_publish_gates_own_quote_check( session_factory: sessionmaker[Session], tmp_path: Path ) -> None: store = LocalSnapshotStore(tmp_path) - catalyst_id = _seed(session_factory, store) - _run(session_factory, store, _Fake()) - (row,) = [c for c in _all(session_factory, Claim) if c.field == claim_field(catalyst_id)] + catalyst_id = seed_catalyst(session_factory, store) + run_readers(session_factory, store, FakeReaders()) + (row,) = [c for c in all_rows(session_factory, Claim) if c.field == claim_field(catalyst_id)] claim = Claim.model_validate(row._asdict()) result = match_quote(claim, document_text(RELEASE, "text/html")) assert result.outcome == "pass", result.reason @@ -303,7 +306,7 @@ def test_a_later_try_that_names_another_disease_for_the_same_sentence_is_refused session_factory: sessionmaker[Session], tmp_path: Path ) -> None: store = LocalSnapshotStore(tmp_path) - catalyst_id = _seed(session_factory, store) + catalyst_id = seed_catalyst(session_factory, store) span = find_quote(document_text(RELEASE, "text/html"), TARGET) assert span is not None with session_factory.begin() as session: # an earlier run kept this sentence with another disease @@ -325,8 +328,54 @@ def test_a_later_try_that_names_another_disease_for_the_same_sentence_is_refused extracted_at=NOW, ) _add(session, earlier) - summary = _run(session_factory, store, _Fake()) + summary = run_readers(session_factory, store, FakeReaders()) assert summary["found"] == 0 - (attempt,) = _all(session_factory, IndicationAttempt) + (attempt,) = all_rows(session_factory, IndicationAttempt) assert (attempt.outcome, attempt.reason, attempt.indication_claim_id) == ("refused", REDRAWN, None) - assert not [c for c in _all(session_factory, ClaimCheck) if c.claim_id == earlier.id] + assert not [c for c in all_rows(session_factory, ClaimCheck) if c.claim_id == earlier.id] + + +def test_with_nothing_to_ask_about_the_mesh_file_is_not_read( + session_factory: sessionmaker[Session], tmp_path: Path, monkeypatch: pytest.MonkeyPatch +) -> None: + def never(*_args: object) -> None: + raise AssertionError("the MeSH file was read") + + monkeypatch.setattr(indicate, "stored_index", never) + report = indicate_catalysts( + session_factory, + LocalSnapshotStore(tmp_path), + _client(FakeReaders()), + limit=10, + now=lambda: NOW, + gate=lambda: [], + ) + assert (report.blocked, report.considered) == ([], 0) + + +@pytest.mark.parametrize( + "error", + [ + SnapshotStoreError("HTTP 503"), + httpx.ConnectError("refused"), + SyntaxError("not XML"), + ValueError("not gzip"), + ], +) +def test_a_mesh_file_that_cannot_be_read_blocks_the_run_and_asks_nothing( + session_factory: sessionmaker[Session], tmp_path: Path, monkeypatch: pytest.MonkeyPatch, error: Exception +) -> None: + store = LocalSnapshotStore(tmp_path) + seed_catalyst(session_factory, store) + + def broken(*_args: object) -> None: + raise error + + monkeypatch.setattr(indicate, "stored_index", broken) + fake = FakeReaders() + report = indicate_catalysts( + session_factory, store, _client(fake), limit=10, now=lambda: NOW, gate=lambda: [] + ) + assert report.blocked + assert "MeSH" in report.blocked[0] + assert fake.sent == [] diff --git a/tests/extract/test_indication_writer.py b/tests/extract/test_indication_writer.py new file mode 100644 index 0000000..5f10727 --- /dev/null +++ b/tests/extract/test_indication_writer.py @@ -0,0 +1,235 @@ +"""The writer's indication pass (ADR 0053, decision 2): plain code, from each catalyst's latest try. + +A confirmed indication is written with a change row citing its claim, so the read API shows it +with its sentence; a later refusal wins over an earlier find; anything that no longer checks out +waits, and nothing is ever unset. +""" + +from datetime import timedelta +from pathlib import Path +from typing import Any +from uuid import uuid4 + +from sqlalchemy import delete, insert, select +from sqlalchemy.orm import Session, sessionmaker + +from catalyst.api.read import published, read_session +from catalyst.db.tables import TABLES +from catalyst.extract.disease import CHECKER_PROMPT, READER_PROMPT +from catalyst.extract.indicate import CONFIRMED_CHECK, claim_field +from catalyst.extract.text import document_text, find_quote +from catalyst.publish.indication import NO_DISEASE, NOT_CONFIRMED, WORDS_NOT_IN_QUOTE, publish_indications +from catalyst.schemas import ( + Catalyst, + CatalystChange, + CheckOutcome, + Claim, + ClaimCheck, + ClaimSubject, + DrugRole, + Indication, + IndicationAttempt, + IndicationResult, + SourceDocument, +) +from catalyst.snapshots import LocalSnapshotStore +from extract.test_indicate import NOW, RELEASE, TARGET, FakeReaders, all_rows, run_readers, seed_catalyst + + +def _publish(factory: sessionmaker[Session], store: LocalSnapshotStore) -> dict[str, Any]: + return publish_indications(factory, store, lambda: NOW).summary() + + +def _detail(factory: sessionmaker[Session], catalyst_id: Any) -> Any: + with read_session(factory) as session: + return next(d for d in published(session) if d.id == catalyst_id) + + +def test_a_confirmed_indication_is_written_and_shown_in_its_sources_own_words( + session_factory: sessionmaker[Session], tmp_path: Path +) -> None: + store = LocalSnapshotStore(tmp_path) + catalyst_id = seed_catalyst(session_factory, store) + run_readers(session_factory, store, FakeReaders()) + assert _publish(session_factory, store) == {"indications_written": 1, "indications_held": {}} + detail = _detail(session_factory, catalyst_id) + assert detail.indication is not None + assert detail.indication.name == "melanoma" # the sentence's word, not the MeSH name "Melanoma" + (source,) = [s for s in detail.sources if "indication_id" in s.fields] + assert source.quote == TARGET + changes = [c for c in all_rows(session_factory, CatalystChange) if c.field == "indication_id"] + assert [c.new_value for c in changes] == [str(detail.indication.id)] + + +def test_a_second_pass_writes_nothing_more(session_factory: sessionmaker[Session], tmp_path: Path) -> None: + store = LocalSnapshotStore(tmp_path) + seed_catalyst(session_factory, store) + run_readers(session_factory, store, FakeReaders()) + _publish(session_factory, store) + assert _publish(session_factory, store) == {"indications_written": 0, "indications_held": {}} + + +def test_a_later_refusal_wins_over_an_earlier_find( + session_factory: sessionmaker[Session], tmp_path: Path +) -> None: + store = LocalSnapshotStore(tmp_path) + catalyst_id = seed_catalyst(session_factory, store) + run_readers(session_factory, store, FakeReaders()) + (found,) = all_rows(session_factory, IndicationAttempt) + later = IndicationAttempt( + catalyst_id=catalyst_id, + claim_id=found.claim_id, + outcome=IndicationResult.REFUSED, + reason="the second reader did not confirm the sentence states the disease", + reader_prompt_version=READER_PROMPT[1], + checker_prompt_version="9.9.9", + reader_model=found.reader_model, + checker_model=found.checker_model, + attempted_at=NOW + timedelta(days=1), + ) + with session_factory.begin() as session: + session.execute(insert(TABLES[IndicationAttempt]).values(**later.model_dump())) + assert _publish(session_factory, store)["indications_written"] == 0 + (row,) = [c for c in all_rows(session_factory, Catalyst) if c.id == catalyst_id] + assert row.indication_id is None + + +def test_without_the_second_readers_pass_the_indication_waits( + session_factory: sessionmaker[Session], tmp_path: Path +) -> None: + store = LocalSnapshotStore(tmp_path) + seed_catalyst(session_factory, store) + run_readers(session_factory, store, FakeReaders()) + with session_factory.begin() as session: + session.execute(delete(TABLES[ClaimCheck]).where(TABLES[ClaimCheck].c.check_name == CONFIRMED_CHECK)) + assert _publish(session_factory, store) == { + "indications_written": 0, + "indications_held": {NOT_CONFIRMED: 1}, + } + + +def test_an_unreadable_copy_holds_the_indication( + session_factory: sessionmaker[Session], tmp_path: Path +) -> None: + store = LocalSnapshotStore(tmp_path) + seed_catalyst(session_factory, store) + run_readers(session_factory, store, FakeReaders()) + with session_factory() as session: + key = session.execute(select(TABLES[SourceDocument].c.snapshot_key)).scalar_one() + (tmp_path / key).unlink() + summary = _publish(session_factory, store) + assert summary["indications_written"] == 0 + assert summary["indications_held"] == {"the stored copy is missing": 1} + + +def test_two_catalysts_with_one_disease_share_one_indication_row( + session_factory: sessionmaker[Session], tmp_path: Path +) -> None: + store = LocalSnapshotStore(tmp_path) + seed_catalyst(session_factory, store, n=1) + seed_catalyst(session_factory, store, n=2) + run_readers(session_factory, store, FakeReaders()) + assert _publish(session_factory, store)["indications_written"] == 2 + assert len(all_rows(session_factory, Indication)) == 1 + + +def test_an_attempt_pointing_at_another_catalysts_claim_waits( + session_factory: sessionmaker[Session], tmp_path: Path +) -> None: + store = LocalSnapshotStore(tmp_path) + first = seed_catalyst(session_factory, store, n=1) + second = seed_catalyst(session_factory, store, n=2, role=DrugRole.IN_COMBINATION) + run_readers(session_factory, store, FakeReaders()) + (found,) = [a for a in all_rows(session_factory, IndicationAttempt) if a.catalyst_id == first] + stolen = IndicationAttempt( + catalyst_id=second, + claim_id=found.claim_id, + outcome=IndicationResult.FOUND, + indication_claim_id=found.indication_claim_id, + reader_prompt_version=READER_PROMPT[1], + checker_prompt_version=CHECKER_PROMPT[1], + reader_model="another", + checker_model="another", + attempted_at=NOW, + id=uuid4(), + ) + with session_factory.begin() as session: + session.execute(insert(TABLES[IndicationAttempt]).values(**stolen.model_dump())) + summary = _publish(session_factory, store) + assert summary["indications_written"] == 1 + assert summary["indications_held"] == {"the claim is not this catalyst's indication claim": 1} + + +def _plant(factory: sessionmaker[Session], catalyst_id: Any, value: dict[str, Any]) -> None: + """Store a found try whose claim says `value`, as no reader would: to test the writer's own checks.""" + span = find_quote(document_text(RELEASE, "text/html"), TARGET) + assert span is not None + with factory.begin() as session: + document = session.execute(select(TABLES[SourceDocument].c.id).limit(1)).scalar_one() + event = session.execute( + select(TABLES[Claim].c.id).where(TABLES[Claim].c.field == "catalyst:1").limit(1) + ).scalar_one() + claim = Claim( + source_document_id=document, + subject_type=ClaimSubject.CATALYST, + subject_id=catalyst_id, + field=claim_field(catalyst_id), + value={"text_version": "1"} | value, + quote=span.quote, + quote_start=span.start, + quote_end=span.end, + agent="extractor", + prompt_id=READER_PROMPT[0], + prompt_version=READER_PROMPT[1], + provider="fake", + model="planted", + extracted_at=NOW, + ) + check = ClaimCheck( + claim_id=claim.id, check_name=CONFIRMED_CHECK, outcome=CheckOutcome.PASS, checked_at=NOW + ) + attempt = IndicationAttempt( + catalyst_id=catalyst_id, + claim_id=event, + outcome=IndicationResult.FOUND, + indication_claim_id=claim.id, + reader_prompt_version=READER_PROMPT[1], + checker_prompt_version=CHECKER_PROMPT[1], + reader_model="planted", + checker_model="planted", + attempted_at=NOW, + ) + for row in (claim, check, attempt): + session.execute(insert(TABLES[type(row)]).values(**row.model_dump())) + + +def test_words_the_quote_does_not_hold_are_never_written( + session_factory: sessionmaker[Session], tmp_path: Path +) -> None: + store = LocalSnapshotStore(tmp_path) + catalyst_id = seed_catalyst(session_factory, store) + _plant( + session_factory, + catalyst_id, + {"mesh_id": "D008545", "mesh_name": "Melanoma", "words": "heart failure"}, + ) + summary = _publish(session_factory, store) + assert summary == {"indications_written": 0, "indications_held": {WORDS_NOT_IN_QUOTE: 1}} + + +def test_one_bad_row_is_held_and_the_rest_are_written( + session_factory: sessionmaker[Session], tmp_path: Path +) -> None: + store = LocalSnapshotStore(tmp_path) + good = seed_catalyst(session_factory, store, n=1) + bad = seed_catalyst( + session_factory, store, n=2, role=DrugRole.IN_COMBINATION + ) # left alone by the readers + run_readers(session_factory, store, FakeReaders()) + _plant(session_factory, bad, {"mesh_id": "not-a-mesh-id", "mesh_name": "Melanoma", "words": "melanoma"}) + summary = _publish(session_factory, store) + assert summary["indications_written"] == 1 + assert summary["indications_held"] == {NO_DISEASE: 1} + rows = {c.id: c.indication_id for c in all_rows(session_factory, Catalyst)} + assert rows[good] is not None + assert rows[bad] is None diff --git a/tests/test_cli.py b/tests/test_cli.py index 8e673b8..73a6983 100644 --- a/tests/test_cli.py +++ b/tests/test_cli.py @@ -17,8 +17,10 @@ from catalyst.collectors.coverage import CoverageEntry, Listing from catalyst.collectors.sec.press_release import NoPressReleaseError from catalyst.extract.evaluate import Scores +from catalyst.extract.indicate import IndicationReport from catalyst.extract.run import DocumentResult, ExtractionReport from catalyst.links import LinkReport, LinkResult, LinkStatus +from catalyst.publish.indication import IndicationWrites from catalyst.publish.run import DecisionReport from catalyst.publish.sanity import SafeReport from catalyst.publish.write import PublishReport @@ -118,6 +120,13 @@ def _safe_to_write_job_stands_in(monkeypatch: pytest.MonkeyPatch) -> None: monkeypatch.setattr(cli, "check_safe_to_write", lambda *_args: SafeReport(claims=1)) # pyright: ignore[reportUnknownLambdaType, reportUnknownArgumentType] +@pytest.fixture(autouse=True) +def _indication_jobs_stand_in(monkeypatch: pytest.MonkeyPatch) -> None: + """Stand in for the indication job and the writer's indication pass, which need a database.""" + monkeypatch.setattr(cli, "indicate_catalysts", lambda *_args, **_kw: IndicationReport()) # pyright: ignore[reportUnknownLambdaType, reportUnknownArgumentType] + monkeypatch.setattr(cli, "publish_indications", lambda *_args, **_kw: IndicationWrites()) # pyright: ignore[reportUnknownLambdaType, reportUnknownArgumentType] + + class _FakeEngine: def dispose(self) -> None: pass @@ -1038,3 +1047,58 @@ def summary(self) -> dict[str, Any]: def test_the_indication_job_refuses_a_limit_below_one(limit: str) -> None: with pytest.raises(SystemExit): main(["indicate-catalysts", "--limit", limit]) + + +def _extract_stands_in(monkeypatch: pytest.MonkeyPatch, published: Any) -> None: + monkeypatch.setenv("CATALYST_ANTHROPIC_API_KEY", "test-key") + monkeypatch.setattr(cli, "create_engine", lambda *_args: _FakeEngine()) # pyright: ignore[reportUnknownLambdaType, reportUnknownArgumentType] + monkeypatch.setattr(cli, "extract_documents", lambda *_args, **_kw: ExtractionReport()) # pyright: ignore[reportUnknownLambdaType, reportUnknownArgumentType] + monkeypatch.setattr(cli, "resolve_claims", lambda *_args: MatchReport()) # pyright: ignore[reportUnknownLambdaType, reportUnknownArgumentType] + monkeypatch.setattr(cli, "run_checks", lambda *_args: CheckReport()) # pyright: ignore[reportUnknownLambdaType, reportUnknownArgumentType] + monkeypatch.setattr(cli, "run_second_reading", lambda *_args, **_kw: SecondReadingReport()) # pyright: ignore[reportUnknownLambdaType, reportUnknownArgumentType] + monkeypatch.setattr(cli, "decide_claims", lambda *_args: DecisionReport()) # pyright: ignore[reportUnknownLambdaType, reportUnknownArgumentType] + monkeypatch.setattr(cli, "publish_claims", published) + + +def test_a_failed_indication_read_keeps_every_summary_still_writes_and_fails_the_run( + monkeypatch: pytest.MonkeyPatch, capsys: pytest.CaptureFixture[str] +) -> None: + _extract_stands_in(monkeypatch, lambda *_args, **_kw: PublishReport()) # pyright: ignore[reportUnknownLambdaType, reportUnknownArgumentType] + wrote: list[bool] = [] + + def broken(*_args: object, **_kw: object) -> IndicationReport: + raise SnapshotStoreError("Supabase Storage refused the MeSH file: HTTP 503") + + def write(*_args: object, **_kw: object) -> IndicationWrites: + wrote.append(True) + return IndicationWrites(written=1) + + monkeypatch.setattr(cli, "indicate_catalysts", broken) + monkeypatch.setattr(cli, "publish_indications", write) + assert main(["extract-catalysts", "--limit", "3", "--snapshot-store", "local"]) == 1 + printed = json.loads(capsys.readouterr().out) + assert "error" in printed["indications"]["read"] + assert printed["indications"]["indications_written"] == 1 + assert wrote == [True] + assert "published" in printed + + +def test_no_indication_is_asked_for_when_publishing_did_not_finish( + monkeypatch: pytest.MonkeyPatch, capsys: pytest.CaptureFixture[str] +) -> None: + def broken(*_args: object, **_kw: object) -> PublishReport: + raise ValueError("bad row") + + _extract_stands_in(monkeypatch, broken) + asked: list[bool] = [] + monkeypatch.setattr(cli, "indicate_catalysts", lambda *_args, **_kw: asked.append(True)) # pyright: ignore[reportUnknownLambdaType, reportUnknownArgumentType] + assert main(["extract-catalysts", "--limit", "3", "--snapshot-store", "local"]) == 1 + printed = json.loads(capsys.readouterr().out) + assert "skipped" in printed["indications"] + assert asked == [] + + +@pytest.mark.parametrize("limit", ["0", "-1"]) +def test_the_extract_run_refuses_a_limit_below_one(limit: str) -> None: + with pytest.raises(SystemExit): + main(["extract-catalysts", "--limit", limit])