diff --git a/docs/adr/0001-local-first-bootstrap-runtime.md b/docs/adr/0001-local-first-bootstrap-runtime.md new file mode 100644 index 00000000..82459f70 --- /dev/null +++ b/docs/adr/0001-local-first-bootstrap-runtime.md @@ -0,0 +1,32 @@ +# ADR 0001: durable local-first bootstrap runtime + +Status: accepted for implementation in P01. + +NetRatel will have four durable, API-owned states: `Unconfigured`, +`Configuring`, `Ready`, and `RecoveryRequired`. State is held in a protected +descriptor and transition journal on persistent API-owned storage; it is never +inferred from an empty user table, a missing file, migration history, or a +failed database connection. + +Before `Ready`, the API exposes only liveness, setup status, safe built-in +branding, and setup operations authenticated with a deployment-controlled, +single-use bootstrap proof. Business APIs, agent admission, schedulers, +outbox work, Akka authority and optional integrations are not started merely +to fail later. `RecoveryRequired` is the outcome when a selected store cannot +be used after initialization, not a reason to create a new SQLite database or +reopen ownership setup. + +Setup provisions and migrates the selected store before accepting an initial +administrator password. It commits the stable principal, tenant, initialization +marker, and non-secret setup data atomically in the selected provider, then +activates the descriptor using the same operation identifier. A restart that +observes the committed marker completes activation without duplicating an owner. + +An established PostgreSQL/OIDC deployment is adopted only by an explicit, +idempotent compatibility check for meaningful existing application state and +verified identity mappings. A migrated-but-empty database is never adoption +evidence. Runtime composition is selected once at startup, not per request. + +Configuration precedence is deployment-owned configuration, durable +administrator/setup values where allowed, then product defaults. Deployment +examples are not deployment-owned values. diff --git a/docs/adr/0002-stable-principals-and-scoped-permissions.md b/docs/adr/0002-stable-principals-and-scoped-permissions.md new file mode 100644 index 00000000..a6408b7e --- /dev/null +++ b/docs/adr/0002-stable-principals-and-scoped-permissions.md @@ -0,0 +1,27 @@ +# ADR 0002: stable principals and scoped permissions + +Status: accepted for implementation in P02 and P03. + +Every interactive identity resolves to a durable application principal ID. +An external principal is linked only by verified issuer and subject; a native +principal is linked to its local Identity account. Email is profile/login data, +not an account-link or authorization key. Existing verified OIDC links remain +valid through adoption and upgrade. + +Effective authority is evaluated as a tuple of principal, permission, tenant, +resource, and (when present) credential and operation policy. A selected +tenant may narrow an existing grant but cannot manufacture one. The API is the +authority for every protected operation; Web access projections control +navigation only. + +P03 will replace administrator-group-only operational decisions with a single +effective-access service used by local users, OIDC users, integration +credentials, and delegated MCP execution. It will persist explicit role/scope +assignments and fail closed for unmapped protected endpoints. Built-in roles +are reconciled idempotently; custom roles are limited by the acting user's +delegation ceiling. Existing OIDC group mappings are preserved through an +explicit compatibility adapter, not recreated on every sign-in. + +Role, membership, disablement and credential changes invalidate or revalidate +server-side authority on the next operation and have defined stream behavior. +They do not rely indefinitely on stale browser claims. diff --git a/docs/adr/0003-provider-selection-and-migration-ownership.md b/docs/adr/0003-provider-selection-and-migration-ownership.md new file mode 100644 index 00000000..5338927c --- /dev/null +++ b/docs/adr/0003-provider-selection-and-migration-ownership.md @@ -0,0 +1,27 @@ +# ADR 0003: provider selection and migration ownership + +Status: accepted for implementation in P04. + +NetRatel will support an explicit provider choice: SQLite for a single-node +deployment, bundled PostgreSQL, or externally managed PostgreSQL. Provider +selection is independent of authentication mode. An initialized descriptor +binds the instance to its selected provider; a connection failure never causes +fallback to another provider. + +PostgreSQL migration history remains intact. SQLite has its own explicit +migration assembly/snapshot and provider-specific model/query implementations +where required. Setup, the migrations container, the API and design-time tools +must resolve the same provider and migration owner. Upgrade tests use real +migrations, never `EnsureCreated` or an in-memory substitute. + +The current PostgreSQL model contains `pg_trgm`, GIN/trigram indexes, `jsonb`, +SQL defaults and PostgreSQL filters. P04 will retain their PostgreSQL semantics +and add honest SQLite equivalents for search, ordering, uniqueness, +concurrency, outbox/idempotency and durable command/job state. Unsupported +SQLite multi-instance topology is rejected; shared network-volume and +cross-host SQLite are not advertised. + +Changing an existing provider is a separate documented data-migration process, +not an automatic setup option. Backup and restore cover application/identity +state, bootstrap state, Data Protection material, agent signing material and +branding assets. diff --git a/docs/adr/0004-local-http-mcp-delegation.md b/docs/adr/0004-local-http-mcp-delegation.md new file mode 100644 index 00000000..48b20ec9 --- /dev/null +++ b/docs/adr/0004-local-http-mcp-delegation.md @@ -0,0 +1,23 @@ +# ADR 0004: local HTTP MCP delegation + +Status: accepted for implementation in P06 through P08. + +NetRatel retains distinct browser, local API credential, HTTP MCP ingress, +short-lived MCP execution, OIDC/M2M, system, and native-agent trust paths. +Token prefixes or routing hints select a candidate scheme only; each scheme +performs its own cryptographic and purpose validation. + +P06 introduces separate local API and HTTP MCP credentials. P08 extends the +existing HTTP MCP resource boundary rather than creating a parallel API bypass. +The gateway validates the ingress credential's purpose, audience and resource, +then obtains caller-bound delegated execution authority. The API evaluates the +intersection of current principal access, credential ceiling, tenant/resource +scope, target/environment policy, confirmation/approval and idempotency rules +before side effects. + +Gateway persistence records only the authorization data needed for current +revocation and auditability. It never substitutes a shared administrator or +agent credential. Invalid credentials, insufficient access, rate limits and +dependency failures remain distinguishable without leaking account existence. +Existing external-OIDC HTTP MCP configuration and resource names remain +compatible. diff --git a/docs/adr/0005-effective-branding-precedence.md b/docs/adr/0005-effective-branding-precedence.md new file mode 100644 index 00000000..9554d4fc --- /dev/null +++ b/docs/adr/0005-effective-branding-precedence.md @@ -0,0 +1,20 @@ +# ADR 0005: effective branding and configuration precedence + +Status: accepted for implementation in P09 and P10. + +NetRatel is the immutable upstream product and protocol identity. Branding can +change only visible deployment presentation: display names, approved assets, +colours, URLs and related copy. It never changes assemblies, packages, +database schema identities, signing audiences, cookies, Data Protection +application names, agent protocols or MCP resource identifiers. + +For each field the effective value is deployment-managed configuration first, +durable administrator override second where the field is editable, and the +approved NetRatel default last. The administration UI shows deployment-managed +values but cannot overwrite them. Stored overrides are kept separately from +effective values so a reset removes only the stored override. + +Theme selection remains client presentation state. P10 will execute the small +preference decision before visible content is painted and will test computed +colours before application runtime for light, dark and system choices, +including unavailable storage and custom branding. diff --git a/docs/implementation/local-first-endpoint-permissions.md b/docs/implementation/local-first-endpoint-permissions.md new file mode 100644 index 00000000..42e690da --- /dev/null +++ b/docs/implementation/local-first-endpoint-permissions.md @@ -0,0 +1,42 @@ +# Local-first endpoint and permission inventory + +Baseline inspected at `7ef86801c80b6cdf3c495ce7ff0a4fff41fa8ac4` for +[P00](https://github.com/BostonTechnologies/netratel/issues/30). This is the +authoritative starting inventory for P03; it is deliberately an inventory, not +evidence that the current administrator-group policies meet scoped-RBAC needs. + +## Current boundary + +`NetRatel.API/Program.cs` defaults all otherwise-unannotated endpoints to an +authenticated **Operator** claim (role/group). Explicit policies are +`Operator`, `McpOperatorPolicyAdmin`, `AkkaShadowAccess`, +`ClientArtifactsWrite`, `ClientArtifactsUpload`, `ClientArtifactsDownload`, +`HealthRead`, `M2MOnly`, `AgentAccess`, `AgentGatewayAccess`, and +`MachineTokenApi`. The API also has OIDC, machine-token, M2M, system and native +agent schemes. This provides a useful separation of trust paths but does not +yet provide a durable local principal or a shared tenant/resource evaluator. + +## Route families and target checks + +| Route family | Current endpoint policy | Current target boundary | P03 permission family / acceptance owner | +| --- | --- | --- | --- | +| `/api/v2/tenants`, tenant cards and tenant-scoped lists | `Operator` / fallback | Route `tenantId` plus service queries | Tenant administration; inventory/list/search isolation | +| `/api/v2/agents/{tenantId}/{agentId}` including telemetry, logs, presence and streams | `Operator` | Tenant and agent IDs; gateway services | Client inventory, telemetry/log read, streaming revalidation | +| Agent commands, tasks, jobs, scripts and schedules | `Operator` | Tenant/agent IDs; command/job services | Script edit/execute, command, job/schedule/task with approval/idempotency preserved | +| Files, artifacts, downloads and uploads | `Operator`, artifact read/write policies, or `M2MOnly` | Tenant/agent/artifact ownership | File read/write/delete and artifact/update publication | +| Terminals and remote support | `Operator` or `M2MOnly` | Tenant/agent/session ownership; existing transport checks | Terminal and remote-support permissions with current target policy retained | +| MCP operator client, file, observability, command, script, job, task and request routes | mostly `M2MOnly` | Existing MCP policy/profile, confirmation, idempotency and target checks | Integration-management plus operation-specific effective access; no gateway bypass | +| MCP policy administration | `McpOperatorPolicyAdmin` | Persisted policy/profile IDs | MCP policy administration and audit | +| Development MCP/onboarding and operator-target routes | `Operator` | Tenant/agent and grant IDs | Enrollment/client-management and scoped target administration | +| Agent enrollment, refresh, updates and gateway transport | `AgentAccess` / `AgentGatewayAccess` | Native agent identity and enrollment state | Native Client identity remains separate; no operator credential reuse | +| Client artifacts and update publication | explicit artifact policies | Artifact/release ownership | Artifact/update publication; preserve native updater contract | +| Health, OpenAPI and operational endpoints | `HealthRead`, fallback, or explicit anonymous metadata where present | No business resource | Instance administration / deliberate pre-ready status only | + +## P03 enforcement contract + +P03 adds a testable endpoint-registration inventory that fails for a protected +route without an explicit permission mapping or an approved non-business +exception. It tests direct object reads, lists, counts, search, downloads, +SSE/WebSocket paths, gateway admission, command dispatch and scheduled or +background execution. Any existing confirmation, target, environment, +approval, or idempotency restriction remains an additional constraint. diff --git a/docs/implementation/local-first-progress.md b/docs/implementation/local-first-progress.md new file mode 100644 index 00000000..59bdb187 --- /dev/null +++ b/docs/implementation/local-first-progress.md @@ -0,0 +1,122 @@ +# Local-first implementation progress + +Umbrella: [#29](https://github.com/BostonTechnologies/netratel/issues/29) + +Integration branch: `main` + +Baseline: `7ef86801c80b6cdf3c495ce7ff0a4fff41fa8ac4` (`chore(deps): modernize September 2026 dependency stack`) + +This ledger records durable evidence only. Raw logs, credentials, screenshots +and disposable environments remain outside the repository. + +## Accepted programme scope + +The programme adds local-first setup, native accounts, scoped RBAC, SQLite, +guided setup, purpose-separated integration credentials, CLI/stdio/HTTP MCP +support, deployment branding, correct first paint, API documentation and a +verified prerelease. It preserves established PostgreSQL/OIDC installs, native +Client credentials and workflows, the current dependency baseline, approved +brand defaults, and receipt-backed release provenance. + +It excludes production/fleet changes, private configuration or consumer-lock +changes, dependency rollbacks, destructive migration rewrites, tag movement, +asset replacement, package-visibility mutation, and repository-rule bypass. + +## Phase ledger + +| Phase | Issue | PR | Merge SHA | Evidence / current state | +| --- | --- | --- | --- | --- | +| P00 | [#30](https://github.com/BostonTechnologies/netratel/issues/30) | [#43](https://github.com/BostonTechnologies/netratel/pull/43) | pending | Inventory and ADRs on `docs/issue-30-local-first-inventory`; local baseline is green; hosted exact-head CI is pending. | +| P01 | [#31](https://github.com/BostonTechnologies/netratel/issues/31) | pending | pending | Blocked by P00 merge. | +| P02 | [#32](https://github.com/BostonTechnologies/netratel/issues/32) | pending | pending | Blocked by P01 merge. | +| P03 | [#33](https://github.com/BostonTechnologies/netratel/issues/33) | pending | pending | Blocked by P02 merge. | +| P04 | [#34](https://github.com/BostonTechnologies/netratel/issues/34) | pending | pending | Blocked by P03 merge. | +| P05 | [#35](https://github.com/BostonTechnologies/netratel/issues/35) | pending | pending | Blocked by P04 merge. | +| P06 | [#36](https://github.com/BostonTechnologies/netratel/issues/36) | pending | pending | Blocked by P03/P04/P05 merges. | +| P07 | [#37](https://github.com/BostonTechnologies/netratel/issues/37) | pending | pending | Blocked by P06 merge. | +| P08 | [#38](https://github.com/BostonTechnologies/netratel/issues/38) | pending | pending | Blocked by P06/P07 merges. | +| P09 | [#39](https://github.com/BostonTechnologies/netratel/issues/39) | pending | pending | Blocked by P03/P05 merges. | +| P10 | [#40](https://github.com/BostonTechnologies/netratel/issues/40) | pending | pending | Blocked by P09 merge. | +| P11 | [#41](https://github.com/BostonTechnologies/netratel/issues/41) | pending | pending | Blocked by P07/P08/P10 merges. | +| P12 | [#42](https://github.com/BostonTechnologies/netratel/issues/42) | pending | pending | Remains open through verified publication. | + +## P00 inventory + +### Runtime and persistence + +- .NET SDK is pinned to `10.0.401`; root product version is `0.1.0-rc.2` in + `Directory.Build.props` and `release/release-manifest.json`. +- `AddNetRatelInfrastructure` currently requires + `ConnectionStrings:NetRatelDb` or `Default` and unconditionally calls + `UseNpgsql`. `OrchestratorDbContext` owns the current EF model/migrations; + it contains PostgreSQL extension, index, `jsonb`, filter and SQL-default + assumptions. `NetRatel.Migrations` is the explicit migration component. +- API persists a Data Protection key ring (default `.keys`, application name + `NetRatel-Keyring`) and separately loads agent signing material. Existing + OIDC signing records, agent credentials/refresh tokens, tenant data, + outbox, command/job, MCP policy and audit state are in the application + database and are upgrade-critical. +- API currently starts storage initialization, hosted seed/retention/catalog, + outbox and search services plus configured Akka authority before it can + distinguish a setup-needed instance. P01 owns safe runtime gating. + +### Identity, transport and presentation + +- Browser/API authentication currently supports provider-neutral OIDC (with + Azure compatibility aliases), optional machine tokens, M2M, system tokens + and native agent tokens. The default/fallback operational policy is an OIDC + administrator role/group assertion; there is no persisted local user model. +- Web is a browser-session client of the API, while the API remains the + authorization and enrollment authority. Native Clients use separate + enrollment, credential, refresh and signing paths. CLI/stdio use configured + OIDC M2M credentials; HTTP MCP has its own resource/delegation boundary. +- `App.razor` currently loads theme preference JavaScript after body content; + P10 owns first-paint correction. The approved NetRatel brand pack is already + merged; P09 adds controlled effective overrides without changing identities. + +### Release and CI baseline + +- `v0.1.0-rc.1` is the only remote release tag. `rc.2` is an untagged source + candidate and must be re-inventoried in P12 before selecting the final RC. +- Public PR validation builds/tests, creates final component images and runs + source/release-image generic OIDC smoke. The non-publishing release workflow + produces archives, SBOMs, checksums and attestations. Promotion requires an + immutable receipt, preflight, journal, exact digest tests and explicit + prerelease publication; it must remain so. +- Baseline hosted evidence: [PR #28](https://github.com/BostonTechnologies/netratel/pull/28) + merged after all listed Public PR validation checks passed. P00 local command + results are added below when complete; historical counts are not reused as + current evidence. + +## Architecture decisions and test ownership + +| Concern | Decision | Implementation phase | Primary evidence | +| --- | --- | --- | --- | +| Bootstrap / legacy adoption | [ADR 0001](../adr/0001-local-first-bootstrap-runtime.md) | P01 | concurrent/replay/restart/adoption real-store tests | +| Stable principal / access | [ADR 0002](../adr/0002-stable-principals-and-scoped-permissions.md) | P02/P03 | local/OIDC and two-tenant direct/stream tests | +| Provider / migrations | [ADR 0003](../adr/0003-provider-selection-and-migration-ownership.md) | P04 | SQLite/PostgreSQL migration, restart and restore tests | +| HTTP MCP delegation | [ADR 0004](../adr/0004-local-http-mcp-delegation.md) | P06–P08 | joined gateway/API and concurrent-scope tests | +| Branding / configuration | [ADR 0005](../adr/0005-effective-branding-precedence.md) | P09/P10 | options/UI and before-runtime computed-colour tests | +| Protected routes | [endpoint inventory](local-first-endpoint-permissions.md) | P03 | unmapped-route gate plus target-scope matrix | + +## Commands and validation + +| Commit | Command | Result | +| --- | --- | --- | +| `7ef8680` | remote tag/release, issue/PR and branch-protection inventory | Completed 2026-09-20; `v0.1.0-rc.1` is the only remote tag; release inventory is recorded above. | +| `7ef8680` | `dotnet restore NetRatel.sln` | Passed. Restoring was required because prior ignored local test assets were stale and selected VSTest despite the repository's Microsoft.Testing.Platform setting. | +| `7ef8680` | `dotnet build NetRatel.sln --configuration Release --no-restore` | Passed: 0 errors, 42 pre-existing warnings. | +| `7ef8680` | `dotnet test NetRatel.sln --configuration Release --no-build --filter 'category!=compose' -- --report-trx --report-trx-filename 'netratel-p00-{asm}_{tfm}_{arch}.trx'` | Passed: 1,860 succeeded, 4 explicit live-environment skips, 0 failed. TRX is retained only as local CI-style evidence. | + +## Current checkpoint + +Current phase: P00. Branch: `docs/issue-30-local-first-inventory`. PR: +[#43](https://github.com/BostonTechnologies/netratel/pull/43). Next action: +wait for required checks on the current head, merge through the protected path, +then update this ledger and begin P01 from refreshed `main`. + +## Blockers + +None currently identified. Publication permissions and package ownership will +be verified only in P12 against the selected candidate; no release action has +been attempted.