Skip to content

feat(dogfood): automate the five-application evidence loop - #36

Merged
howlcipher merged 1 commit into
mainfrom
feat/dogfood-five-application-run
Aug 20, 2026
Merged

howlcipher merged 1 commit into
mainfrom
feat/dogfood-five-application-run

Conversation

@howlcipher

Copy link
Copy Markdown
Owner

Summary

  • Adds a 5-application dogfood cohort to the dashboard: start a run, confirm real applications through the normal Assisted Apply flow, and get an automatic sanitized report with a rule-based go/no-go verdict once the fifth is captured.
  • Reuses existing durable data (job_funnel, assisted_fill_summary, application_preflight, human_interactions, GetHumanEffortMetrics's join pattern) rather than duplicating it. The only new persistence is two small tables — dogfood_cohorts and dogfood_cohort_applications — for cohort membership and the operator's own closed-vocabulary feedback (no answer text, no PII).
  • Capture happens inside ConfirmAssistedSubmission's existing transaction, so it is atomic with the APPLIED write: a closed browser, an abandoned session, or the automatic pipeline's separate APPLIED path can never be miscounted. ConfirmAssistedSubmission's signature changes from error to (int, error) to report the capture ordinal back to the caller.
  • One optional one-question feedback prompt appears only right after a dogfood-captured confirmation (never on a normal confirm), and is always skippable.
  • Verdict logic is deterministic, not an LLM call: any "filled incorrectly" feedback → pause for correctness; any other friction category repeated ≥2 times → name it and recommend one fix; otherwise → keep using it, stop coding. The report only ever recommends one task — it never opens a backlog item or writes code itself.
  • Replaces the manual documentation/dogfood_scorecard.md workflow (left in place as historical record of #545).

Fixed two incidental issues found while wiring this in:

  • cmd/dashboard/main_test.go's hand-rolled applied_jobs fixture was missing UNIQUE(url), which production has and ConfirmAssistedSubmission's ON CONFLICT(url) depends on.
  • cmd/dashboard/schema_upgrade_test.go's prepareDashboardSchema — a second startup-ensure-list deliberately kept in sync with main() — was missing EnsureDogfoodSchema and the new tables in its assertion list.

Test plan

  • gofmt -l ./cmd ./pkg ./internal — clean
  • go build ./...
  • go vet ./...
  • go test ./... — all pass except TestServeKnowledgeProfile_KeepsBackupsBoundedAndPrivate, a pre-existing timestamp-collision flake confirmed to fail identically on unmodified main
  • npm run lint / npm test (85 tests) / npm run build in cmd/dashboard/ui
  • Manual: end-to-end cohort lifecycle exercised via TestDogfoodRun_EndToEnd (start → confirm 5 → feedback → auto-complete → report)

🤖 Generated with Claude Code

Replaces the manual dogfood_scorecard.md workflow with an in-dashboard
5-application cohort: start a run, confirm real applications as usual,
answer one optional feedback question per confirmation, and get an
automatic sanitized report with a rule-based go/no-go verdict once the
fifth is captured.

Reuses existing durable data (job_funnel, assisted_fill_summary,
application_preflight, human_interactions) rather than duplicating it;
the only new persistence is cohort membership and the operator's own
closed-vocabulary feedback, captured atomically inside
ConfirmAssistedSubmission's existing transaction so a closed browser or
an automatic-pipeline APPLIED transition can never be miscounted.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
@howlcipher
howlcipher merged commit df7bbf3 into main Aug 20, 2026
1 of 3 checks passed
@howlcipher
howlcipher deleted the feat/dogfood-five-application-run branch August 20, 2026 02:11
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant