docket-lens

structured analysis of federal comment dockets. pulls every filing from ECFS (or takes a directory of filings you already have — pdf, docx, txt), classifies each one against the docket's defining categories, and produces a categorized dataset, a flat csv, and a sourced narrative report of what the record actually says.

built during my participation in FCC PS Docket 26-189 (the "military-grade" drone proposal), where it processed 680 filings. the design should transfer to any ECFS docket and, with schema changes, to regulations.gov dockets.

see it in action

reports/ has the actual output from the 26-189 run: narrative reports from successive reruns as the record grew, and a short brief. RUNBOOK.md has every tuned threshold with the evidence that set it, plus measured costs and runtimes.

what it does

four scripts, end to end:

fcc_download.py: acquisition (ECFS-specific, optional). pulls every filing with an attachment for a given docket from the public ECFS API into downloads/. handles Akamai's bot-blocking (plain requests get a 403), paginates the filing list, and is rerun-safe: it skips files already on disk, so you can re-pull a docket daily through a comment deadline and only new filings cost a request. if you already have filings from somewhere else, skip this script; docket_triage.py only cares about the directory.

docket_triage.py: extraction. walks a directory of filings and produces one structured record per unique document:

• text extraction: pdftotext (pypdf fallback), docx2txt, raw txt. likely-scanned pdfs are flagged for OCR instead of silently misread
• form-letter dedup: 8-gram shingle sets, jaccard > 0.85 collapses copies onto an exemplar, so a 40-copy campaign costs one model call and gets counted, not reread
• llm categorization against a fixed json schema (see below), model-agnostic: anthropic api or any openai-compatible endpoint (ollama, llama.cpp, vllm)
• idempotent: records keyed on content hash + schema version. reruns process only new files, so you can re-pull the docket daily through a deadline
• outputs: results_v3.jsonl (raw records), filings.csv (flat), and a printed digest (positions, filer types, per-category stances, arguments)

docket_report.py: reporting. an agentic writing pass that reads the csv plus a seeded selection of full filings and produces a sourced narrative report of the record:

• seeded read list, not model free choice: the notice itself, every supporting filing, every trade association, plus per-category top filings by evidence with filer-type diversity. the model may add reads but logs a rationale per extra read
• the writer receives the record's per-category proportions with an instruction to allocate coverage proportionally; conclusions about which fight is "sharpest" must be earned from cited content, not inherited from the reading diet
• provenance: claims in the report carry source-comment markers back to the underlying filings
• lint + repair loop on the output: a style check (no em-dashes) and a salience check (per-category mention share vs record share, warns past 1.5x divergence either way). violations are quoted back to the model for targeted rewrite, max two iterations, then hard-warn

docket_brief.py: condensing (optional). one cheap api call turns the newest report into a ~700-word brief: no filings are re-read and no tool calls are made, since the long report already did that work. kept quotes must travel with their source citation and are re-verified against the downloads, same lint/repair loop as the report.

docket_triage.py and docket_report.py are the core: extraction and reporting. fcc_download.py handles ECFS retrieval when you don't already have the filings; docket_brief.py is a convenience wrapper that condenses a report already produced. See RUNBOOK.md for exact rerun commands, every tuned threshold with the evidence behind it, and measured results.

quickstart

pip install -r requirements.txt
# poppler's pdftotext improves extraction quality over the pypdf fallback;
# it's a system package (not pip-installable), so install it separately if
# you want it - triage runs fine without it

export ANTHROPIC_API_KEY=sk-ant-...     # required for docket_report.py and
                                         # docket_brief.py; docket_triage.py
                                         # also accepts this
# docket_triage.py ONLY: any openai-compatible local endpoint instead
export LLM_BASE_URL=http://localhost:11434/v1 LLM_MODEL=<model>

python fcc_download.py                          # optional: pull filings from ECFS -> ./downloads
python docket_triage.py ./downloads             # extraction + digest
python docket_report.py                         # narrative report -> reports/report_YYYY.MM.DD.md
python docket_brief.py                          # short brief -> reports/brief_YYYY.MM.DD.md

input expectations

ECFS bulk downloads name files {submissionid}_{attachment}_{name}. note that _2_ and higher attachments exist (exhibits). the triage step does not merge a filing's attachments into one record: each file is analyzed and counted independently, so a filing with 3 attachments produces 3 records. download every attachment anyway (fcc_download.py does this by default) since exhibits often carry the substantive argument the cover letter doesn't; just budget for per-attachment, not per-filing, counts when reading filings.csv.

the schema (v3)

per filing: filer name and type, overall position, cross-cutting arguments, remedies requested, data citation flag, notable checkable claims, and a two-sentence summary. the core is category_positions: for every category the docket defines, a mandatory {stance, arguments, note} object where silence is an explicit stance. all categories get the identical question with a category-agnostic argument vocabulary.

that symmetry is a bias fix, not a nicety. see below.

methodology and limitations

• classifications are llm-generated. for anything that matters, verify against the source pdfs; i hand-checked every claim i published or filed
• bias history, disclosed: v1 of this schema had a dedicated attacks_category_4 field and none for other categories, because i built it while advocating on category (4). the resulting report over-covered lidar about 2x relative to the record. v3's symmetric schema plus the salience lint corrected it, measurably: residual coverage deviations now track newsworthiness, not my stake. if you adapt this tool while holding a position, assume your schema inherits it until you measure
• the salience lint uses term lists, which are fragile. my first version counted every occurrence of "docket" as a docking-station mention. use word boundaries and negative lookaheads, and eyeball the lint before trusting it
• dedup threshold (0.85 jaccard) collapses form letters but keeps lightly-edited variants as unique; tune per docket
• scanned pdfs are flagged, not OCRed. run ocrmypdf and re-drop them

disclaimers

not legal advice. i filed comments in the docket this was built on; classifications and reports are my analysis of public filings, not the FCC's. all underlying documents are public record on ECFS.