From cb7a2ba0293f143f05321a02edac490d23781a11 Mon Sep 17 00:00:00 2001 From: PARTH ROHIT Date: Sat, 26 Sep 2026 11:52:06 +0100 Subject: [PATCH 1/2] docs: bring the docs up to date with v0.3.0 and the dev-to-main release process (#491) Corrects the capability entry for registration (development mode is open, no email verification, no pre-approved address), records the shared unresolved-reference classifier and the stack-metadata reconciliation, the worker's commit-between-facts and failed-analysis purge, UTC timestamps, GitHub tree URLs and the Ollama egress guidance. Rewrites CONTRIBUTING's release section as promote, verify, sync back, tag from main, and adds the v0.3.0 changelog row, the storage and retention doc to the index and the cleanup script to the scripts index. --- CHANGELOG.md | 1 + CONTRIBUTING.md | 39 +++++++++++++------ README.md | 6 +-- apps/backend/app/extraction/support_matrix.py | 2 +- docs/CAPABILITIES.md | 2 +- docs/DEVELOPMENT.md | 5 ++- docs/README.md | 1 + docs/architecture/REPOSITORY_INTELLIGENCE.md | 6 +-- docs/architecture/SYSTEM_OVERVIEW.md | 23 +++++------ docs/security/AI_PROVIDER_EGRESS.md | 6 ++- scripts/README.md | 1 + 11 files changed, 60 insertions(+), 32 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 52733c20..a0087690 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -12,5 +12,6 @@ Unreleased work lives on `dev`. | Version | Date | Summary | Notes | | --- | --- | --- | --- | +| [v0.3.0](https://github.com/Second-Origin/PARTHA/releases/tag/v0.3.0) | 2026-09-26 | Reliability and truthfulness: Engineering Review no longer reports references into external code as findings (894 to 33 on the FastAPI template), cancel works on SQLite, explicit-UTC timestamps, clearer first-run errors, FastAPI decorator/router guards recognised, failed analyses no longer leave facts on disk, and removal of a pre-approved registration email (see the release notes for the security details). | [Release notes](https://github.com/Second-Origin/PARTHA/releases/tag/v0.3.0) · [`v0.2.0…v0.3.0`](https://github.com/Second-Origin/PARTHA/compare/v0.2.0...v0.3.0) | | [v0.2.0](https://github.com/Second-Origin/PARTHA/releases/tag/v0.2.0) | 2026-09-09 | Sealed `ri.v1` snapshots as the single read model; evidence-backed extractors and deterministic resolution; Repository Lineage (RFC-0002); accounts, owner isolation, and optional OAuth; the marketing site; and a documentation accuracy pass. | [Release notes](https://github.com/Second-Origin/PARTHA/releases/tag/v0.2.0) · [`v0.1.0…v0.2.0`](https://github.com/Second-Origin/PARTHA/compare/v0.1.0...v0.2.0) | | [v0.1.0](https://github.com/Second-Origin/PARTHA/releases/tag/v0.1.0) | 2026-07-09 | First public release: repository ingestion and analysis, the Repository Intelligence engine, architecture / dependency / documentation / review surfaces, the AI provider foundation, and the export pipeline. | [Release notes](https://github.com/Second-Origin/PARTHA/releases/tag/v0.1.0) | diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index f10044ad..61090bc4 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -480,21 +480,35 @@ code-owner review, passing required status checks, and no deletion or force-push **Cutting a release.** -1. Bump the version in `package.json`, `apps/frontend/package.json`, - `apps/marketing/package.json`, `apps/backend/pyproject.toml`, and the FastAPI - `version=` in `apps/backend/app/main.py`, together with any documentation that - is genuinely out of date. Merge that to `dev` like any other change. -2. Open a `dev` → `main` pull request and merge it once checks are green. -3. Tag the release commit `vMAJOR.MINOR.PATCH` and push the tag. -4. Once the GitHub Release exists and its notes are curated (below), add a row for - the version to [`CHANGELOG.md`](CHANGELOG.md) — a one-line summary and a link to - the release notes — so the in-repo index stays current. +A release is a promotion, in this order: + +1. **Prepare on `dev`.** Bump the version in `package.json`, `apps/frontend/package.json`, + `apps/marketing/package.json`, both `package-lock.json` files, `apps/backend/pyproject.toml`, + and the FastAPI `version=` in `apps/backend/app/main.py`; point any version-bearing copy + (the marketing site's "Try PARTHA vX.Y.Z" button) at the new version; and update any + documentation that is genuinely out of date. Merge that to `dev` like any other change. +2. **Promote.** Open a `dev` → `main` pull request and merge it once every check is green. + Use a merge commit, not a squash, so `main` and `dev` keep shared history. +3. **Verify.** Check through the GitHub API, not just local git, that `main` still exists and + points at the merge commit (`gh api repos///branches/main`), and that the + Production deployment for that commit succeeded. Anything user-facing (the marketing + copy) is live only from this point. +4. **Sync back.** The promotion leaves `main` one merge commit ahead of `dev`. Bring it back + with a pull request into `dev` from a **throwaway branch created at `main`'s tip** + (for example `sync/main-into-dev`), never from `main` itself (see the warning below). +5. **Tag from `main`.** Tag the release commit on `main` `vMAJOR.MINOR.PATCH` and push the tag. +6. **Curate the release** (below), then add a row for the version to + [`CHANGELOG.md`](CHANGELOG.md), a one-line summary and a link to the release notes, so + the in-repo index stays current. Pushing a `v*` tag runs [`.github/workflows/release.yml`](.github/workflows/release.yml): a `validate` job re-runs the release-relevant frontend and backend checks against real PostgreSQL and Redis, and only if that passes does a `github-release` job create the GitHub Release with `gh release create --generate-notes` — an automatically generated -list of merged pull requests and new contributors. +list of merged pull requests and new contributors. **Let the workflow create the release** +rather than creating it by hand first: with the release already present, `github-release` +fails with "tag_name already exists" (the validation still runs and passes, but the run +shows red). Those generated notes are a **baseline, not the final notes.** After the workflow runs, a maintainer edits the release (`gh release edit --notes-file …`, or the Releases @@ -505,7 +519,10 @@ done until that curation has happened. `main` is only ever a pull-request **target**. Never open a pull request whose *source* branch is `main`: merged head branches are deleted automatically, which would delete -`main` itself. +`main` itself. This is not hypothetical: merging the step-4 sync-back with `main` as its head +did exactly that for v0.3.0, and `main` had to be recreated at the promotion commit. If it +happens, `gh api -X POST repos///git/refs -f ref=refs/heads/main -f sha=` +restores it (the branch ruleset reapplies), and step 3's API check is how you notice. PARTHA is pre-1.0. Minor versions may change behaviour; the release notes say so when they do. diff --git a/README.md b/README.md index fdceb769..84194ed8 100644 --- a/README.md +++ b/README.md @@ -43,7 +43,7 @@ Each surface reads the sealed snapshot for the analysed revision: - **Repository Intelligence** — builds an immutable, revision-addressed structural snapshot from supported repository sources, with evidence and provenance on supported facts. - **Architecture** — an interactive graph of modules and resolved relationships, plus an evidence-cited authentication explanation for supported Python/FastAPI patterns. - **Dependency Graph** — direct declarations from `package.json`, `pyproject.toml`, and `requirements.txt`, with resolved pins from two lockfile formats recorded as resolutions, never as direct edges. -- **Engineering Review** — findings that are each backed by a stored evidence span; unassessed categories stay visible. No score, grade, or health percentage. +- **Engineering Review** — findings that are each backed by a stored evidence span; unassessed categories stay visible. References into third-party or platform code, local names and non-code assets are not reported as findings. No score, grade, or health percentage. - **Repository Insights** — defined counts, ratios, diagnostics, and coverage from one snapshot. No change-over-time claims. - **Repository Lineage** — repeated imports of the same repository and branch grouped into a durable history, browsable through the API and UI. This is revision *history*, not cross-revision comparison. - **Documentation & export** — structural documentation, and Review / Documentation / Architecture / Dependencies exported through one JSON / Markdown / HTML / PDF pipeline. @@ -129,7 +129,7 @@ The [development guide](docs/DEVELOPMENT.md) covers the full test / lint / build - **Optional AI can be external.** Depending on configuration, AI calls a configured provider; only local providers keep everything on the host. See the [egress policy](docs/security/AI_PROVIDER_EGRESS.md). - **In-process worker.** One daemon worker thread inside the API process handles one analysis job at a time; there is no separate worker service or job queue. -Non-auth product routes require authentication, repository access is owner-scoped, provider keys are Fernet-encrypted at rest, and AI egress is validated against a deployment-owned allowlist with DNS pinning — meaningful controls, but not a claim of production hardening. See [SECURITY.md](SECURITY.md) and [`docs/CAPABILITIES.md`](docs/CAPABILITIES.md) for the details. +Non-auth product routes require authentication, repository access is owner-scoped, provider keys are Fernet-encrypted at rest, and AI egress is validated against a deployment-owned allowlist with DNS pinning — meaningful controls, but not a claim of production hardening. Registration does not verify email ownership, and in the default `development` environment any address may register. See [SECURITY.md](SECURITY.md) and [`docs/CAPABILITIES.md`](docs/CAPABILITIES.md) for the details. ## Documentation @@ -152,7 +152,7 @@ Issues and pull requests are welcome. Start with [CONTRIBUTING.md](CONTRIBUTING. ## Releases -`main` carries the latest tagged release; `dev` is where active development happens and is the target of every pull request. PARTHA is pre-1.0, so minor versions may change behaviour — each release's notes say what moved. See [all releases](https://github.com/Second-Origin/PARTHA/releases), the [changelog](CHANGELOG.md), and [CONTRIBUTING § Releases](CONTRIBUTING.md#14-releases). +`main` carries the latest tagged release; `dev` is where active development happens and is the target of every pull request. A release is a promotion: prepare on `dev`, promote `dev` to `main`, verify, sync back, then tag from `main`. PARTHA is pre-1.0, so minor versions may change behaviour — each release's notes say what moved. See [all releases](https://github.com/Second-Origin/PARTHA/releases), the [changelog](CHANGELOG.md), and [CONTRIBUTING § Releases](CONTRIBUTING.md#14-releases). ## License diff --git a/apps/backend/app/extraction/support_matrix.py b/apps/backend/app/extraction/support_matrix.py index d40567bf..a5172f3f 100644 --- a/apps/backend/app/extraction/support_matrix.py +++ b/apps/backend/app/extraction/support_matrix.py @@ -852,7 +852,7 @@ def _code_list(items: tuple[str, ...]) -> str: "authentication-isolation", "Authentication and owner isolation", PublicStatus.IMPLEMENTED, - "Email/password, Argon2, short-lived access tokens, rotating refresh tokens with reuse detection. Google and GitHub OAuth sign-in and account linking are implemented but inert until provider credentials are configured, and never create an account. Registration is gated by an admin-managed email allowlist in every environment, except the first account on an empty instance. Protected resources are owner-scoped; non-owner access returns 404.", + "Email/password, Argon2, short-lived access tokens, rotating refresh tokens with reuse detection. Google and GitHub OAuth sign-in and account linking are implemented but inert until provider credentials are configured, and never create an account. Registration is gated by an admin-managed email allowlist in every environment except `development` (the default when running from source), where any address may register, and except the first account on an empty instance, which becomes its owner. Registration does not verify email ownership, so an approved address can be claimed by whoever registers it first; there is no admin role or in-app approval. Protected resources are owner-scoped; non-owner access returns 404.", ("product.authentication-isolation",), ), PublicCapability( diff --git a/docs/CAPABILITIES.md b/docs/CAPABILITIES.md index ea75291f..3d2341b7 100644 --- a/docs/CAPABILITIES.md +++ b/docs/CAPABILITIES.md @@ -31,7 +31,7 @@ release; `dev` carries active development. | --- | --- | --- | | Archive upload and public GitHub import | **Implemented** | ZIP/TAR-family archives and shallow public GitHub HTTPS clones; size and path-safety limits apply. Private GitHub cloning and other repository hosts are not supported. | | Repository explorer | **Implemented** | Owner-scoped file tree plus bounded text/image preview, binary detection, and truncation. | -| Authentication and owner isolation | **Implemented** | Email/password, Argon2, short-lived access tokens, rotating refresh tokens with reuse detection. Google and GitHub OAuth sign-in and account linking are implemented but inert until provider credentials are configured, and never create an account. Registration is gated by an admin-managed email allowlist in every environment, except the first account on an empty instance. Protected resources are owner-scoped; non-owner access returns 404. | +| Authentication and owner isolation | **Implemented** | Email/password, Argon2, short-lived access tokens, rotating refresh tokens with reuse detection. Google and GitHub OAuth sign-in and account linking are implemented but inert until provider credentials are configured, and never create an account. Registration is gated by an admin-managed email allowlist in every environment except `development` (the default when running from source), where any address may register, and except the first account on an empty instance, which becomes its owner. Registration does not verify email ownership, so an approved address can be claimed by whoever registers it first; there is no admin role or in-app approval. Protected resources are owner-scoped; non-owner access returns 404. | | Analysis lifecycle | **Implemented** | Database-backed, cancellable job with progress, bounded retry, lease renewal, and stale-worker recovery. | | Repository Intelligence | **Implemented with disclosed limits** | Immutable, revision-addressed `ri.v1` snapshots with normalized facts, evidence, query APIs, and a total canonical graph hash. Semantic extraction is strongest for supported Python and TypeScript/JavaScript constructs. | | Repository lineage | **Implemented with disclosed limits** | Repeated imports of the same repository and branch are grouped into a durable, owner-scoped lineage with duplicate-revision detection (RFC-0002). `GET /repositories/{id}/lineage` returns the ordered history and the repository detail page renders it. Refresh and cross-revision comparison on top of a lineage are not built. | diff --git a/docs/DEVELOPMENT.md b/docs/DEVELOPMENT.md index 3af49288..025dc619 100644 --- a/docs/DEVELOPMENT.md +++ b/docs/DEVELOPMENT.md @@ -55,7 +55,10 @@ npm run dev:frontend ``` Open `http://localhost:5173`. There is no seeded account or sample -repository — register a new local account through the UI, then add a +repository (and no pre-approved email address) — register a new local account through the UI +(in the default `APP_ENV=development` any address is accepted; anywhere else the first account +on an empty instance becomes its owner and every later one must be added with +`apps/backend/scripts/approve_email.py`), then add a repository (upload an archive or import a public GitHub repository over HTTPS) and start analysis. diff --git a/docs/README.md b/docs/README.md index 28083c44..dd6b4270 100644 --- a/docs/README.md +++ b/docs/README.md @@ -20,6 +20,7 @@ are point-in-time evidence, not current-state documentation. | [SECURITY](../SECURITY.md) | Anyone reporting a vulnerability | How to disclose privately. Never open a public issue for a vulnerability. | | [AI provider egress policy](security/AI_PROVIDER_EGRESS.md) | Operators and backend contributors | Deployment-owned provider destination policy, DNS pinning, redirect handling, safe defaults, and required production network controls. | | [Connecting an AI provider](operations/AI_PROVIDER_SETUP.md) | Anyone enabling the optional AI workspace | The end-to-end setup path (Settings and the `ai/*` API), per-provider requirements, the egress-policy prerequisite for a local or custom Ollama endpoint, Ollama's slow-first-request and one-at-a-time behaviour, and a troubleshooting table. | +| [Storage and retention](operations/STORAGE_AND_RETENTION.md) | Operators | What grows on disk, what PARTHA discards on its own (failed and cancelled analyses), what is left to you (older revisions), and the `cleanup_storage.py` report/cleanup script. | | [Database migration rehearsal and recovery](operations/DATABASE_MIGRATION_REHEARSAL.md) | Operators and backend contributors | Disposable Alembic rehearsal command, supported baseline evidence, production preflight, and truthful restore-based recovery decisions. | | [WCAG 2.2 AA accessibility baseline](accessibility/WCAG_2_2_AA_BASELINE.md) | Frontend contributors and accessibility reviewers | Reproducible automated coverage for the Phase 0 journeys, the outstanding human verification checklist, confirmed findings, and linked follow-up issues. | | [CODE_OF_CONDUCT](../CODE_OF_CONDUCT.md) | Everyone | Expected conduct and how to report a violation. | diff --git a/docs/architecture/REPOSITORY_INTELLIGENCE.md b/docs/architecture/REPOSITORY_INTELLIGENCE.md index 5ad42cd7..4f6c9c55 100644 --- a/docs/architecture/REPOSITORY_INTELLIGENCE.md +++ b/docs/architecture/REPOSITORY_INTELLIGENCE.md @@ -230,10 +230,10 @@ speculative edges. | Consumer | Module | Reads | | --- | --- | --- | -| Architecture | `app/analysis/architecture.py` | exclusively the sealed snapshot query layer — nodes, resolved edges, `classified_as` assertions, diagnostics, and evidence. An `RI-RES-UNRESOLVED` whose target is a declared dependency or the language platform is not treated as a module relationship gap. No legacy `repo_metadata['intelligence']` read. | +| Architecture | `app/analysis/architecture.py` | exclusively the sealed snapshot query layer — nodes, resolved edges, `classified_as` assertions, diagnostics, and evidence. An `RI-RES-UNRESOLVED` is judged by the shared classifier (`app/insights/relationship_diagnostics.py`, `classify_unresolved`): a reference into a declared dependency or the language platform, a call through a local name, or an import of a non-code asset is not treated as an unmapped relationship; a reference whose in-repo target exists but was not linked (a path alias, a sub-directory package root) still is. No legacy `repo_metadata['intelligence']` read. | | Authentication explanation (#95) | `app/analysis/authentication.py` | exclusively the sealed snapshot query layer — routes, `routes_to`/`injects`/`calls` edges, `classified_as` assertions, diagnostics. No legacy read. | -| Engineering review | `app/review/` | exclusively one sealed snapshot — diagnostics promoted only when an exact same-snapshot fact and evidence span exist; manifest identity; no legacy read and no scores. | -| Repository insights | `app/insights/` | exclusively one sealed snapshot — defined node, relationship, evidence, diagnostic, language, coverage, and extractor counts, with unresolved relationships split into genuine in-repo gaps and expected external references; no legacy read. | +| Engineering review | `app/review/` | exclusively one sealed snapshot — diagnostics promoted only when an exact same-snapshot fact and evidence span exist; manifest identity; no legacy read and no scores. An `RI-RES-UNRESOLVED` is promoted to a finding only when the shared classifier (see Architecture above) calls it a genuine gap, so references into third-party or platform code, local names, non-code assets and in-repo files the analysis merely failed to link are not findings (#467: 894 to 33 findings on `fastapi/full-stack-fastapi-template`, 7,049 to 65 on this repository). The resolver's output is unchanged. | +| Repository insights | `app/insights/` | exclusively one sealed snapshot — defined node, relationship, evidence, diagnostic, language, coverage, and extractor counts, with unresolved relationships split, by the same shared classifier, into genuine in-repo gaps (including in-repo targets the analysis could not link), expected external references, and local names / non-code assets; no legacy read. | | Dependency graph (#158) | `app/graph/` | exclusively one sealed snapshot — `dependency` nodes and resolved `depends_on` edges, with declarations merged across manifests (#156) and lockfile resolutions merged onto the same identity (#209); no legacy read. | | Documentation | `app/services/documentation_service.py` | current-revision sealed projection: observed paths/languages, heuristic roles/modules, routes, dependencies/declarations, diagnostics, and snapshot identity | | AI | `app/ai/repository_context.py` | the same sealed projection; structural facts only, with no source-file contents or fabricated citations | diff --git a/docs/architecture/SYSTEM_OVERVIEW.md b/docs/architecture/SYSTEM_OVERVIEW.md index 157f57b3..a702c07f 100644 --- a/docs/architecture/SYSTEM_OVERVIEW.md +++ b/docs/architecture/SYSTEM_OVERVIEW.md @@ -81,16 +81,16 @@ flowchart LR | `extraction/` | The producer set behind the engine: syntax-aware `python.py` and `typescript.py`, dependency `manifests.py` and `lockfiles.py`, outbound service interactions (`http.py`), Docker Compose resources (`iac.py`), and the `support_matrix.py` that generates the public capability registry. | Persist snapshots or answer product queries. | | `analysis/` | Architecture model — modules, layers, edges, request-flow hints. **Consumer.** | Read the filesystem. | | `graph/` | Dependency graph response model. **Consumer.** | Re-read dependency manifests. | -| `review/` | Deterministic `engineering-review.v2` findings and category assessment over one sealed snapshot. **Consumer.** | Read the legacy JSON model, invent scores/grades, or emit a finding without exact same-snapshot evidence. | +| `review/` | Deterministic `engineering-review.v2` findings and category assessment over one sealed snapshot. An unresolved reference becomes a finding only if the shared classifier in `insights/relationship_diagnostics.py` (also used by Insights and Architecture, #467) calls it a genuine gap. **Consumer.** | Read the legacy JSON model, invent scores/grades, or emit a finding without exact same-snapshot evidence. | | `insights/` | Defined `repository-insights.v1` metrics and breakdowns over one sealed snapshot. **Consumer.** | Read legacy metadata, infer trends without history, or publish undefined health metrics. | | `ai/` | Context builder, prompt builder, orchestrator, provider registry/factory, five provider implementations, and the central egress policy/pinned sender. **Consumer.** | Parse repositories, read source files, or let a tenant expand provider destinations. | | `reports/` | `ReportDocument` intermediate representation, builders, and JSON/Markdown/HTML/PDF renderers. **Second-order consumer** — renders analysis output that already exists. | Re-analyse a repository. | | `auth/` | Argon2 password hashing, HS256 access tokens, rotating refresh tokens with reuse detection. | — | -| `core/` | Settings and validation, database engine, structured logging with redaction, request IDs, metrics, rate limiting, security headers. | — | +| `core/` | Settings and validation, database engine, structured logging with redaction, request IDs, metrics, rate limiting, security headers. Timestamp columns use `models/types.py::UTCDateTime`: stored as UTC and always read back timezone-aware, so the API emits an explicit `Z` on SQLite as on PostgreSQL (#473). | — | | `storage/` | Local filesystem storage for uploads and extracted/cloned repositories. Enforces path safety on extraction. | — | | `workers/control_plane.py` | The queue boundary: which jobs are eligible, and who owns them — claiming, lease renewal, expiry, reclaim, and every ownership guard. | Analyse a repository, or read anything but `analysis_jobs`. | | `workers/runner.py` | The in-process runner: worker identity, the poll/sweep loop, and shutdown. The same loop a standalone worker process would run. | Contain queue policy of its own, or be imported by the API for anything but start/stop. | -| `workers/analysis_worker.py` | Execution of an *already-claimed* job: the extraction pipeline, snapshot sealing, bounded retry, cancellation and stale-job reconciliation. | Decide who owns a job, or serve request-specific data. | +| `workers/analysis_worker.py` | Execution of an *already-claimed* job: the extraction pipeline, snapshot sealing, bounded retry, cancellation and stale-job reconciliation. During the extract stage it commits between facts (about every 0.25 s) so other writers, notably a cancel request, are never locked out of SQLite's single writer for the whole run (#469); it never does this while sealing. On failure or cancellation it discards the build's partial facts (`intelligence/retention.py`, #485). | Decide who owns a job, or serve request-specific data. | --- @@ -154,7 +154,7 @@ contract. Running analysis in a separate process is not implemented today. | --- | --- | --- | | Relational DB | `users`, `refresh_tokens`, `repositories`, `repository_lineages`, `analysis_jobs`, `ai_provider_configs`, `ai_conversation_messages`, the account-access tables (`approved_emails`, `invite_tokens`, `oauth_identities`, `oauth_flow_states`, `oauth_pending_links`, `account_deletion_audits`), `waitlist_entries`, and the normalized `ri_*` snapshot tables | SQLite by default for local development; PostgreSQL is supported through `DATABASE_URL`. Analysis jobs and their worker leases are durable database state. | | `account_deletion_audits` | A minimal, **non-PII** record that an account deletion happened: the deleted user id, status, timestamps, and any failure reason. | Retained after the user row and its data are removed, so a deletion remains auditable without preserving the account. | -| `approved_emails` | The registration gate: which email addresses may create an account. | Registration is allowlisted in **every** environment. The one exception is a genuinely empty instance, where the first account registered is auto-approved and becomes that instance's owner. `invite_tokens` is the superseded invite-code table, retained for historical continuity and no longer a gate. | +| `approved_emails` | The registration gate: which email addresses may create an account. | Registration is allowlisted in every environment **except `development`** (the default `APP_ENV` when running from source), where any address is auto-approved so local use stays frictionless. Outside development the one exception is a genuinely empty instance: the first account registered is auto-approved and becomes that instance's owner. No address is pre-approved by a migration (the seed row `0016` once inserted was removed in #465). Registration does **not verify email ownership**, so an approved address can be claimed by whoever registers it first, and there is no admin role: rows are added with `scripts/approve_email.py`. Account requests accept private-network domains such as `corp.local`. `invite_tokens` is the superseded invite-code table, retained for historical continuity and no longer a gate. | | `oauth_identities`, `oauth_flow_states`, `oauth_pending_links` | Linked Google/GitHub provider identities, in-flight authorization state, and pending link confirmations. | OAuth is inert unless provider credentials are configured. It never creates an account — an unknown identity is returned to the invite-gated registration flow. | | `repository_lineages` | Owner-scoped grouping of repeated imports of the same repository and branch, with 1-based never-reused sequence allocation (RFC-0002). | Uploads and unresolved GitHub imports stay unlineaged: their lineage fields are null and no synthetic lineage is created. | | `waitlist_entries` | Landing-page waitlist signups: normalized email and optional name. | Written by `POST /waitlist`, the one **public write route** in the API. Deliberately unauthenticated; guarded by the `auth` rate-limit class and an existence-concealing response. | @@ -164,7 +164,7 @@ contract. Running analysis in a separate process is not implemented today. | `repositories.file_tree` (JSON column) | The parsed file tree. | Serves the explorer. | | `ai_provider_configs` | One row per user: provider, model, base URL, and the **Fernet-encrypted** API key plus its last four characters. | Owner-scoped; the plaintext key is never stored or returned. A stored endpoint remains unusable unless it satisfies the current deployment egress policy. | | `ai_conversation_messages` | One row per AI Workspace turn: role, content, optional citations, and an explicit `sequence`. | **The AI Workspace thread is durable, not ephemeral.** One ordered thread per owner per repository, so history survives navigating away and returning. `UNIQUE(owner_id, repository_id, sequence)` is the concurrency guard; the repository foreign key cascades, so deleting a repository deletes its turns. A cross-owner read resolves to the same 404 as a missing repository. | -| Filesystem (`STORAGE_PATH`) | Extracted archives and cloned repositories; uploaded archives (deleted after extraction). | Repository source is read from here on demand for file preview. | +| Filesystem (`STORAGE_PATH`) | Extracted archives and cloned repositories; uploaded archives (deleted after extraction). | Repository source is read from here on demand for file preview. What accumulates, what is discarded automatically and the operator cleanup script are in [Storage and retention](../operations/STORAGE_AND_RETENTION.md). | AI provider configuration is **per-user and encrypted at rest**. Each user's API key is encrypted with a Fernet key from `AI_ENCRYPTION_KEY` (required outside `development`/`test`), decrypted only in-process at request time, and injected per request — so a query runs against the caller's own key and bill, never a shared one. @@ -241,7 +241,7 @@ confused. | | `RepositoryResponse.meta` (`RepositoryMeta`) | Repository Intelligence | | --- | --- | --- | -| Source | `RepositoryParser`, at import time | sealed `ri.v1` snapshot, from durable analysis | +| Source | `RepositoryParser`, at import time; language, framework and entry point are then **refreshed from the snapshot when analysis completes** (`ArchitectureAnalyzer.stack_summary`, #475), so an analysed repository's `meta` and its Architecture summary agree | sealed `ri.v1` snapshot, from durable analysis | | How | file-extension `Counter`; `_detect_framework` / `_detect_entry_point` / `_detect_package_manager` / `_detect_license` filename heuristics | evidence-backed extractors + deterministic resolver | | Provenance | none | path + span + producer/version on every fact | | Owner-scoped | via the repository row | yes, in the query service | @@ -257,10 +257,11 @@ explanation, AI context — reads the snapshot query API, never `meta`. import path persists. Its heuristic `meta` is a byproduct of that walk, not a parallel analysis engine, and no consumer under the Repository Intelligence boundary reads it. -*Known UI debt:* `RepositoryDetailPage`, `RepositoriesPage`, and `DashboardPage` render -`meta.language` / `meta.framework` without labelling them as import-time heuristics. Relabelling -those surfaces (or switching them to snapshot-backed values once analysis has completed) is tracked -as follow-up, not done here. +Until analysis completes, `meta.language` / `meta.framework` remain the import-time heuristics, and +the repository pages do not label them as such. Once analysis has completed, the worker replaces them +with the snapshot-derived values where the snapshot supplies one (a value it cannot supply, such as an +unknown framework, keeps the import-time one). Repositories analysed before that change keep their +old values until re-analysed. --- @@ -269,7 +270,7 @@ as follow-up, not done here. | Dependency | Used for | Failure mode | | --- | --- | --- | | `git` (system binary) | Shallow-cloning public GitHub repositories. | Import fails with a normalized external-service error; the partial clone is cleaned up. | -| GitHub (HTTPS) | Source for public repository import. Only `https://github.com/owner/repo` URLs are accepted; no authentication, so no private repositories. | Timeout and size caps abort and clean up. | +| GitHub (HTTPS) | Source for public repository import. Only `https://github.com/owner/repo` URLs are accepted, optionally with `/tree/` (split off as the branch, #481); other page URLs are refused. No authentication, so no private repositories. | Timeout and size caps abort and clean up. | | AI providers | Answering repository questions. Configured per user with an encrypted API key; destinations are centrally policy-checked and DNS-pinned. | A missing, stale, or policy-denied configuration produces a normalized error; the rest of the system is unaffected. | | PostgreSQL, Redis | Optional configured services and CI integration. Redis backs the rate limiter when `RATE_LIMIT_BACKEND=redis`. | Local development uses SQLite and the in-memory rate limiter; neither service is required. | diff --git a/docs/security/AI_PROVIDER_EGRESS.md b/docs/security/AI_PROVIDER_EGRESS.md index fc6f5f92..b236f636 100644 --- a/docs/security/AI_PROVIDER_EGRESS.md +++ b/docs/security/AI_PROVIDER_EGRESS.md @@ -82,7 +82,11 @@ Provider requests explicitly disable redirect following. Any 3xx response is a controlled provider failure; PARTHA never sends a request to its `Location` target. -Policy errors use a generic message. Normal API errors and logs must not include +Policy errors use a generic message. For a configurable endpoint (Ollama) the message +may append one line of guidance that names the administrator setting deciding the +outcome (`AI_EGRESS_ALLOWED_BASE_URLS`, `AI_EGRESS_MODE`, `AI_EGRESS_ALLOWED_CIDRS`), so a +self-hoster is not left stuck (#479); it never carries the URL, address or range. +Normal API errors and logs must not include the rejected URL, hostname, resolved addresses, or any URL credentials. Gemini credentials are sent in the provider-supported API-key header rather than the query string, and HTTPX/httpcore request-detail logging is held at warning level diff --git a/scripts/README.md b/scripts/README.md index 0632c1f6..6d50e6ae 100644 --- a/scripts/README.md +++ b/scripts/README.md @@ -13,6 +13,7 @@ Local workflow helpers. Prefer the root `package.json` scripts for common tasks | `check-capabilities.py` | Validates the capability registry and the generated block in `docs/CAPABILITIES.md` against `app/extraction/support_matrix.py`, so the published capability table cannot drift from the source. `--write` regenerates the block. | CI **Backend** | | `apps/backend/scripts/approve_email.py` | **Operator tool for the registration gate.** Adds an email address to the `approved_emails` allowlist, which is what permits that address to register. There is deliberately no admin UI or API route for this. Approving the same address twice is a harmless no-op. | Maintainer, directly | | `apps/backend/scripts/list_waitlist.py` | Lists landing-page waitlist signups, newest first (`--csv` for a spreadsheet). The owner's review step before deciding whom to approve next. | Maintainer, directly | +| `apps/backend/scripts/cleanup_storage.py` | Reports (default) or, with `--apply`, removes what PARTHA no longer needs: facts left by failed analyses and repository directories with no repository; `--vacuum` shrinks the SQLite file. Never touches a completed analysis. See [Storage and retention](../docs/operations/STORAGE_AND_RETENTION.md). | Operator, directly | | `apps/backend/scripts/rehearse_migrations.py` | Creates disposable SQLite (or explicitly confirmed PostgreSQL) targets, exercises the Alembic clean chain and representative `0004` baseline, then removes the targets. | Maintainer runbook and backend test suite | | `apps/backend/scripts/rehearse_backup_restore.py` | Proves a database + storage backup can be restored into a clean environment byte-for-byte and row-for-row, canonical graph hash included. Accepts no database URL: every target is one it created and removes again, even on failure. PostgreSQL is opt-in and needs `pg_dump`/`pg_restore`. | Maintainer runbook, [migration rehearsal doc](../docs/operations/DATABASE_MIGRATION_REHEARSAL.md) | | `dependency-audit.mjs` | Runs the frontend dependency audit against the policy below and fails on a blocking finding. | CI **Frontend** | From 8e8d60d1da8dda5798a875b55e27b395a934ef0a Mon Sep 17 00:00:00 2001 From: PARTH ROHIT Date: Sat, 26 Sep 2026 13:40:39 +0100 Subject: [PATCH 2/2] docs: add an explicit SemVer versioning policy (VERSIONING.md) (#492) Defines PARTHA's public API, what qualifies for a patch, minor or major release, the pre-1.0 convention (breaking changes bump the minor, patches are bug fixes only), and rules that keep the sequence honest (increment by one, published versions are immutable, one version everywhere). Linked from CONTRIBUTING, GOVERNANCE, SUPPORT, CHANGELOG and the README. --- CHANGELOG.md | 2 +- CONTRIBUTING.md | 6 ++- GOVERNANCE.md | 2 +- README.md | 2 +- SUPPORT.md | 2 +- VERSIONING.md | 121 ++++++++++++++++++++++++++++++++++++++++++++++++ 6 files changed, 129 insertions(+), 6 deletions(-) create mode 100644 VERSIONING.md diff --git a/CHANGELOG.md b/CHANGELOG.md index a0087690..3e6d8862 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -5,7 +5,7 @@ The authoritative, curated notes for each release live on the short index of released versions and links to those notes; it is updated as part of cutting a release ([CONTRIBUTING §14](CONTRIBUTING.md#14-releases)). -PARTHA is pre-1.0. Minor versions may change behaviour; each release's notes say what moved. +PARTHA follows [Semantic Versioning](VERSIONING.md). It is pre-1.0: minor versions may change behaviour; patch versions are bug fixes only. Each release's notes say what moved. Unreleased work lives on `dev`. ## Released diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 61090bc4..684e68ef 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -480,9 +480,11 @@ code-owner review, passing required status checks, and no deletion or force-push **Cutting a release.** -A release is a promotion, in this order: +Which version number a release gets is decided by [VERSIONING.md](VERSIONING.md): review +everything merged since the last tag, take the highest level it reaches (patch, minor or +major), and increment by exactly one step. A release is a promotion, in this order: -1. **Prepare on `dev`.** Bump the version in `package.json`, `apps/frontend/package.json`, +1. **Prepare on `dev`.** Choose the version under the policy, then bump it in `package.json`, `apps/frontend/package.json`, `apps/marketing/package.json`, both `package-lock.json` files, `apps/backend/pyproject.toml`, and the FastAPI `version=` in `apps/backend/app/main.py`; point any version-bearing copy (the marketing site's "Try PARTHA vX.Y.Z" button) at the new version; and update any diff --git a/GOVERNANCE.md b/GOVERNANCE.md index e6ca80c4..33a29b3f 100644 --- a/GOVERNANCE.md +++ b/GOVERNANCE.md @@ -52,7 +52,7 @@ RFC-0002 (Repository Lineage) are the existing examples. ## Releases -Releases are cut by maintainers: bump versions, promote `dev` → `main`, tag `vMAJOR.MINOR.PATCH`. +Releases are cut by maintainers: choose the version under [VERSIONING.md](VERSIONING.md), bump versions, promote `dev` → `main`, tag `vMAJOR.MINOR.PATCH`. The tag triggers [`release.yml`](.github/workflows/release.yml), which re-validates against PostgreSQL and Redis before publishing a GitHub Release; a maintainer then curates the generated notes. Full steps are in [CONTRIBUTING §14](CONTRIBUTING.md#14-releases). PARTHA diff --git a/README.md b/README.md index 84194ed8..c027272b 100644 --- a/README.md +++ b/README.md @@ -152,7 +152,7 @@ Issues and pull requests are welcome. Start with [CONTRIBUTING.md](CONTRIBUTING. ## Releases -`main` carries the latest tagged release; `dev` is where active development happens and is the target of every pull request. A release is a promotion: prepare on `dev`, promote `dev` to `main`, verify, sync back, then tag from `main`. PARTHA is pre-1.0, so minor versions may change behaviour — each release's notes say what moved. See [all releases](https://github.com/Second-Origin/PARTHA/releases), the [changelog](CHANGELOG.md), and [CONTRIBUTING § Releases](CONTRIBUTING.md#14-releases). +`main` carries the latest tagged release; `dev` is where active development happens and is the target of every pull request. A release is a promotion: prepare on `dev`, promote `dev` to `main`, verify, sync back, then tag from `main`. PARTHA follows [Semantic Versioning](VERSIONING.md) and is pre-1.0, so minor versions may change behaviour — each release's notes say what moved. See [all releases](https://github.com/Second-Origin/PARTHA/releases), the [changelog](CHANGELOG.md), and [CONTRIBUTING § Releases](CONTRIBUTING.md#14-releases). ## License diff --git a/SUPPORT.md b/SUPPORT.md index ed4f49eb..3cc20d38 100644 --- a/SUPPORT.md +++ b/SUPPORT.md @@ -20,7 +20,7 @@ PARTHA is **pre-1.0 and under active development.** - Fixes land on `dev` and are included in the next tagged release. Security fixes are provided for `dev` and the latest release (see [SECURITY.md](SECURITY.md#supported-versions)). -- **Minor version bumps may change behaviour.** Each release's notes state what moved; read +- **Minor version bumps may change behaviour** (PARTHA is pre-1.0; see the [versioning policy](VERSIONING.md)); patch releases are bug fixes only. Each release's notes state what moved; read them before upgrading. There is no back-porting to older releases and no long-term-support branch. - When reporting an issue, say which release tag or commit SHA you are on — behaviour on an diff --git a/VERSIONING.md b/VERSIONING.md new file mode 100644 index 00000000..3b35c685 --- /dev/null +++ b/VERSIONING.md @@ -0,0 +1,121 @@ +# Versioning policy + +PARTHA follows [Semantic Versioning 2.0.0](https://semver.org/spec/v2.0.0.html). This document +says what that means here: what counts as PARTHA's public API, what qualifies for a patch, +minor or major release, how the pre-1.0 period works, and the rules that keep the version +sequence honest. The mechanics of cutting a release (promote, verify, sync back, tag) are in +[CONTRIBUTING §14](CONTRIBUTING.md#14-releases); this file decides only **which number**. + +Versions are written `MAJOR.MINOR.PATCH`, tagged `vMAJOR.MINOR.PATCH`, and are plain integers: +`0.9.0` is followed by `0.10.0`, not `1.0.0`. + +## What is the public API + +Semantic Versioning is only meaningful relative to a declared public API. PARTHA's is: + +| Surface | What is covered | +| --- | --- | +| HTTP API | Routes, request and response shapes, status codes and error `code`s in the generated OpenAPI contract (`apps/frontend/src/shared/services/api/generated.ts` is checked against it in CI). | +| Configuration | Environment variables and their defaults and accepted values (`apps/backend/.env.example`, `docs/operations/`). | +| Operator tooling | The command-line interfaces of `apps/backend/scripts/` (`approve_email.py`, `cleanup_storage.py`, ...): names, flags, exit codes. | +| Upgrade path | That `alembic upgrade head` takes a database from any released version to the new one without manual steps, and what data a migration touches. | +| Exports | The structure of the JSON export and the report schema identifiers (`engineering-review.v2`, `repository-insights.v1`). | +| Snapshot schema | `ri.v1` has its own explicit schema version (RFC-0001). A change to it is governed by the RFC, and any incompatible change is at least a **minor** release before 1.0 and a **major** release after. | + +**Not** public API, and free to change in any release: internal Python modules and function +signatures, database table layout beyond the upgrade guarantee above, frontend component +structure and styling, the wording of UI copy and log lines, CI, and the benchmark corpus. + +## What qualifies for each bump + +Decide by reviewing everything merged since the last tag and taking the **highest** level any +change reaches. A release that is mostly fixes but adds one new capability is a minor release. + +### Patch: `0.3.0` -> `0.3.1` + +Backward-compatible **bug fixes only**. Nothing a user or operator could describe as "new". + +- Fixes that make the software do what its documentation already says. +- Security fixes that do not change the public API. +- Dependency updates that change no behaviour. +- Documentation and internal-only changes (refactors, tests, CI). +- A migration only when the fix requires it and it is data-only or corrective (for example, + deleting a row a bad migration inserted). No new tables or columns. + +A patch must **not**: add an endpoint, field, setting, flag, script or UI surface; change a +default; change what an existing response contains beyond correcting a wrong value; or need +any operator action beyond upgrading. + +### Minor: `0.3.0` -> `0.4.0` + +**New backward-compatible functionality**, or a change to existing behaviour that is visible +to users or operators. + +- A new capability: for example Repository Lineage's read API and history page, a new export + format, a new provider. +- New endpoints, response fields, settings, or script options. +- Additive migrations (new tables, columns, indexes). +- Deprecating something (it keeps working; the notes say what replaces it). +- Behaviour changes that are corrections in spirit but observable, such as a different set of + Engineering Review findings or a changed timestamp format. Say so in the notes. + +### Major: `1.x` -> `2.0.0` + +**Incompatible changes** to the public API, after 1.0. + +- Removing or renaming an endpoint, field, setting or script flag; changing what one means. +- A migration that needs manual steps, cannot run on the previous release's data, or discards + data. +- Dropping a supported Python or Node version. +- An incompatible `ri.v1` change (a new schema version is introduced alongside; the old one is + removed only in a major release). + +## Before 1.0 + +SemVer item 4 says anything may change at any time while the major version is `0`. PARTHA +narrows that so that upgraders can still rely on the number: + +- The middle number does the work of "major": **a breaking change bumps the minor version** + (`0.3.0` -> `0.4.0`), and the release notes list it under a **Breaking** heading with what to + do about it. +- The last number stays strictly **bug fixes** (the patch rules above apply unchanged), so + `0.4.1` is always safe to take from `0.4.0`. +- Nothing before 1.0 is guaranteed stable across minor versions, which is why + [SUPPORT.md](SUPPORT.md) says to read each release's notes before upgrading. + +**1.0.0** is a promise, not a milestone: it declares the public API above stable and starts +enforcing the major rule. It will be cut when the maintainers judge that the HTTP API, +configuration and upgrade path are ones they are prepared to keep compatible, and it will say so. +There is no date or feature list attached to it. + +## Rules that keep the sequence honest + +1. **Increment by one, never skip.** Each release differs from the previous tag by exactly one + step: the next patch (`0.3.1`), the next minor with patch reset (`0.4.0`), or the next major. + The sequence `0.1.0, 0.1.1, 0.2.0, ..., 0.9.0, 0.10.0, ..., 1.0.0` is the expected shape. +2. **A published version is immutable.** Never delete, move or re-point a tag or replace a + release's contents. A mistake is corrected by publishing the next version and saying so in its + notes; a bad release may be marked in its own notes as superseded. +3. **One version everywhere.** The release commit sets the same number in `package.json`, + `apps/frontend/package.json`, `apps/marketing/package.json` (and both `package-lock.json` + files), `apps/backend/pyproject.toml` and the FastAPI `version=` in `apps/backend/app/main.py`, + and version-bearing copy such as the marketing "Try PARTHA vX.Y.Z" button. Pull requests + never change a version string. +4. **The notes justify the number.** Release notes group changes by area and state under a + heading which of Security, Breaking, New or Fixes each belongs to, so a reader can check the + bump against this policy. +5. **Pre-releases** use a SemVer pre-release suffix (`0.4.0-rc.1`), are marked pre-release on + GitHub, and are never the target of the "latest" label. +6. **Ties go up.** If a change is arguably a patch and arguably a minor, it is a minor. + +## Version history + +| Version | Date | Kind | Note | +| --- | --- | --- | --- | +| `0.1.0` | 2026-07-09 | first release | The JavaScript packages still read `1.0.0` in this tag (an inconsistency, corrected in 0.2.0). | +| `0.2.0` | 2026-09-09 | minor | Sealed `ri.v1` snapshots, Repository Lineage, accounts. Versions aligned across all packages. | +| `0.3.0` | 2026-09-26 | minor | Reliability and truthfulness release (see the release notes). Published before this policy existed. It is not being re-cut. | + +No release was skipped between these. There has not yet been a patch release; every release so far +has been cut when a batch of work was ready rather than by these rules, which is what this file +now settles.