Find signals that carry no information, and jobs that produce nothing.
A check that fires on 100% of rows is not a check. A cron job that exits 0 and writes nothing is not a job. Both look like coverage from the outside, and neither is -- they fail in the one direction monitoring never looks.
This matters more now that agents write most of the machinery. A model will happily produce a diagnostic that cannot fail, a test that cannot fail, and a confident report about work it never persisted.
Software fails loudly. Data fails quietly.
A column that stopped updating in March. An export that has been byte-identical for six weeks. A nightly job that has "succeeded" every night into an empty file. Nothing alerts on any of it, because nothing is broken — the numbers simply stopped being true, and every dashboard above them kept reporting with complete confidence.
flatline is the judgment: it decides whether a signal still carries information. canary is the trigger: it watches the folders where files land and asks flatline the moment one changes, so the answer arrives before anyone thinks to ask the question.
Neither tool will ever call a file clean that it failed to read. A check that could not run is reported as a failure to check, never as a pass — in code, and with a test, in both tools. Hold us to that one.
Stdlib only, no dependencies. Python 3.11+ (for tomllib).
pip install -e .
flatline --help
Or without installing: python -m flatline --help
Any command takes --report out.html and writes a self-contained HTML file:
no CDN, no fonts, no scripts. It survives being emailed and opened on a machine
that has never heard of this tool, and it styles both light and dark.
flatline --report audit.html jobs
flatline --report deadcode.html code src/
Findings are grouped needs-attention / worth-checking / for-information, each naming the evidence it rests on. A clean run says so explicitly -- and says that a clean run is not the same as not having looked.
python -m flatline scan data/*.csv
python -m flatline scan logs/ --group-by metric
Every column is scored by Shannon entropy. A column carrying 0.00 bits is a constant wearing a signal's name.
Severity depends on what the column claims to be. A constant expiry is ordinary; a
constant flag, status, result, or error_level is a defect, because that column's
whole purpose is to discriminate. Those grade HIGH.
Two things it deliberately does not do:
- Judge thin data. Under
--min-n(default 8) observations it reportsUNDERPOWEREDand declines to conclude. "It never varied across 3 rows" is not evidence. - Confuse absence with agreement. Blank,
None, and whitespace all collapse to one "absent" value, so a never-populated column surfaces asnever populatedrather than quietly looking constant.
Deadness is often conditional. The bug this tool was built from lived in
rh_crosscheck.csv: flag was OK for one metric and DISCREPANCY for another, so
across the whole file it varied and looked healthy. It was constant only within
metric=ntm_oi -- where it had fired on 16 of 16 rows for a week, because the two sides
were measuring different strike bands.
$ python -m flatline scan --group-by metric rh_crosscheck.csv
rh_crosscheck.csv [metric=ntm_oi]
!! flag: CONSTANT
identical value 'DISCREPANCY' on all 16 rows; carries 0.00 bits
-- this column cannot distinguish anything
An always-firing check and a never-firing check are different problems worth telling apart. The first is broken by construction. The second is merely unproven -- you have no evidence it can fire.
Windows only (returns cleanly elsewhere).
python -m flatline jobs
-
EMPTY_WORKDIR(MEDIUM) -- a task with a blankWorkingDirectorystarts in%windir%\system32, and anything it launches inherits that cwd, which makes writes to its own data folder fragile. Scripts that anchor themselves withcd/pushdare not flagged.This is a risk factor, not proof. It was graded HIGH in the first version, on the theory that it explained a real four-day data loss. Measuring an actual machine killed that theory: of 6 flagged tasks, 5 were writing output normally -- including one writing to the same folder as the job that failed. That job had also run fine for three days with the field already blank, so the blank field was constant across both the working and failing periods and cannot be the cause.
Confirm against
STALE_OUTPUT, which is evidence, before acting on this. -
STALE_OUTPUT/MISSING_OUTPUT-- the task ran; a file it should produce didn't change. Pass expected outputs toaudit_job()for this check. -
LAST_RUN_FAILED-- non-zero exit.267009(running) and267011(never run) are correctly not treated as failures.
Exits non-zero on a HIGH finding, for CI.
python -m pytest tests/ -q
52 tests. Two of them exist because they caught real bugs in this code during
development: an alternation order that split PASSED into six single letters, and a
\s+ that let a bare cd match the following line.
Five small tools, one idea: software reports success while doing nothing, and nobody notices for months. Each answers a different question, and each says plainly what it cannot see.
| tool | the question it answers | its blind spot |
|---|---|---|
| attest | did this job run, and did it produce what it claimed? | it sees declared outputs, not whether they are correct |
| canary | what is wrong in the file that just landed? | it never sees whether a job ran at all |
| watchpost | did an output go stale between runs? | it watches files, not the work that made them |
| custody | what did the AI actually do, and was it right? | it cannot see inside the model, and never claims an answer was true |
They share one hash chain and one signature implementation, imported rather than copied — two versions of a trust primitive diverge the first time only one gets fixed.
Why there are several of these, and when we will delete one — the rule each tool had to pass to exist, the one overlap that is real, and the date we have committed to settling it.
MIT. Built by Automated Workflow.