Skip to content

0.2.0: visibility and control - #7

Closed
iamitp wants to merge 7 commits into
mainfrom
claude/evaluate-progress-MmAkA
Closed

iamitp wants to merge 7 commits into
mainfrom
claude/evaluate-progress-MmAkA

Conversation

@iamitp

@iamitp iamitp commented Apr 21, 2026

Copy link
Copy Markdown
Owner

Summary

0.2.0 is a durability-for-daily-use cycle sitting before the public launch. 0.1.0 worked; 0.2.0 makes the four primitives trustworthy for week-over-week use. Seven commits, each leaving tests green (47 passing, up from 24).

  • Cost tracking. vera.llm.chat() now returns (text, usage). Every chat / chat-retry / audit call writes a USAGE entry to the provenance JSONL with tokens + estimated USD. vera status prints Spend (30d): $X over N calls when any USAGE entries exist.
  • Visible regenerate. A rule violation used to trigger a silent second call. Now prints ⟲ regenerating: <violation> and, if the rule layer disagrees twice, appends a ⚠ rule layer disagreed twice footer instead of silently accepting the second draft.
  • vera prune. Moves transcripts older than N days (mtime, default 90) into MEMORY_DIR/_archive/, always retaining the most-recent --keep (default 50). Supports --dry-run. vera status surfaces archive counts.
  • Smarter memory summary. load_memory_summary() now layers deduped RULE / OBSERVED captures from the last 30 days on top of headlines from the last 5 transcripts. No mid-word truncation.
  • Audit controls. --last N, --since YYYY-MM-DD, --model MODEL, plus VERA_AUDIT_MODEL env var.
  • Lenient VERA-CAPTURE parsing. Accepts blockquote >, list -/*, arbitrary indent, case-insensitive keys, and blank-line separators. Malformed blocks produce a console warning plus a MALFORMED provenance entry instead of silent drops.
  • Install drift cleanup. README / LAUNCH / install.sh / READY-TO-PASTE now all say pipx install vera-clerk (PyPI) rather than the stale git+https://github.com/iamitp/vera.

Version bump 0.1.0 → 0.2.0. Not publishing to PyPI in this cycle — that ships when the launch is greenlit.

Test plan

  • pytest -q locally — 47 passed.
  • python3 -m build + python3 -m twine check dist/* in a fresh venv — both PASSED.
  • Smoke vera init --quick && vera chat with a real ANTHROPIC_API_KEY on a clean shell. Confirm:
    • ⟲ regenerating: prints when a rule violation is caught.
    • After Ctrl-D, vera status shows the spend line.
    • vera audit --last 1 --model claude-haiku-4-5-20251001 respects the override.
    • vera prune --older-than 30 --dry-run lists candidates; without --dry-run moves them into _archive/.

Planned in /root/.claude/plans/evaluate-where-we-are-glistening-waffle.md.

claude added 7 commits April 21, 2026 06:41
README, LAUNCH, install.sh, and READY-TO-PASTE were still pointing
at the git+https install. vera-clerk has been live on PyPI since
0.1.0; the ship.sh patching step was never run. Align the four
surfaces now so visitors see the canonical install command.
chat() now returns (text, usage). USD_PER_MTOK carries public rates
for the handful of models Vera actually dispatches to; unknown models
get usd=None rather than a fabricated number. log_usage() appends a
USAGE kind to the provenance JSONL per call (chat / chat-retry /
audit). vera status reads the last 30 days of USAGE entries and
prints 'Spend (30d): $X over N calls' when any exist.

Hidden cost is the fastest route to a deleted tool; this makes it
visible without coupling Vera to a particular billing endpoint.
Rule violations previously triggered a silent second call; the user
saw only a longer wait and a different response. Now we:

- print '⟲ regenerating: <violation>' (dim) before the retry so the
  user knows why the turn took extra time
- if attempt 2 also violates, append a '⚠ rule layer disagreed twice'
  footer to the response so the user knows the model ignored the
  rule layer rather than silently pretending it complied

Hidden retries kill trust. Seeing the rule layer work is the point.
Two linked changes so memory stays useful past month one.

vera prune [--older-than N --keep N --dry-run]
  Move transcripts older than N days (mtime, default 90) into
  MEMORY_DIR/_archive/, always retaining the most-recent --keep
  (default 50). --dry-run lists targets without moving. vera status
  surfaces 'Archive: N transcripts' when the dir exists.

load_memory_summary()
  Previously last 10 files × first 200 chars (mid-word truncation,
  no durability). Now two layers:
    1. Durable facts: dedup of RULE / OBSERVED provenance entries
       from the last 30 days (bullet per entry, 160-char cap).
    2. Recent transcripts: the last 5 transcripts, one-line headline
       each.
  Falls back cleanly when the log / dir are missing.
vera audit previously hardcoded [-5:] and the provider's default
audit_model. Users with deeper history or cheaper-model preferences
had no lever. This adds three levers:

  --last N           audit last N transcripts by mtime (default 5)
  --since YYYY-MM-DD only transcripts on/after this date
  --model MODEL      override the audit model for this run

plus a VERA_AUDIT_MODEL env var read in detect_provider() so power
users can set their preferred auditor once per shell. run_audit()
grows matching kwargs; --since uses mtime rather than filename so
copied or hand-placed transcripts still work.
The previous parser required the marker and field lines in exact
form — unindented, no prefix, three lines. Real model outputs wrap
captures in markdown blockquotes, list bullets, or extra whitespace,
and those silently dropped.

Now parse_captures:
  - strips blockquote '>', list '-'/'*', and arbitrary indent before
    the marker and each field
  - accepts case-insensitive 'kind:' / 'content:'
  - scans up to 6 lines after the marker (was 3)
  - tolerates a blank line between marker and fields
  - returns (captures, warnings); the chat loop prints a dim notice
    and logs a MALFORMED provenance entry per warning so the user
    knows something was dropped instead of silent data loss.

Adds MALFORMED to ProvenanceKind. Seven new regression tests cover
blockquote, list, indentation, case, blank-line-separator cases.
0.2.0 ships 'visibility and control': cost tracking, visible
regenerate, vera prune + smarter memory summary, audit controls,
lenient VERA-CAPTURE parsing, and the install-drift cleanup.

47 tests (+23 since 0.1.0). Build + twine check clean in a fresh
venv (local debian-pinned 'packaging' rejects the 2.4 Metadata
header; CI uses fresh tooling).
@iamitp iamitp mentioned this pull request Apr 21, 2026
@iamitp iamitp closed this Apr 21, 2026

iamitp commented Apr 21, 2026

Copy link
Copy Markdown
Owner Author

Superseded by #8. The 0.2.0 durability cycle worked (47 tests passing), but the wind-down decision makes it moot — Vera is archiving at 0.1.1 rather than shipping 0.2.0. The seven feature commits remain in branch history for reference.


Generated by Claude Code

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants