feat(framework): introduce typed product framework and Trino reference operator

.github/workflows/framework-runtime.yml

@@ -0,0 +1,134 @@
+name: Framework Runtime (ARM64)
+
+# Runtime acceptance uses pinned third-party product images and its own kind
+# cluster. PR generation/unit/lint checks remain in test.yml and lint.yml.
+on:
+  workflow_dispatch:
+
+permissions:
+  contents: read
+
+concurrency:
+  group: framework-runtime-${{ github.ref }}
+  cancel-in-progress: false
+
+jobs:
+  formal-runtime:
+    runs-on: ubuntu-24.04-arm
+    timeout-minutes: 90
+    defaults:
+      run:
+        shell: bash
+        working-directory: operator-go
+    env:
+      KIND_VERSION: v0.32.0
+      KUBECTL_VERSION: v1.36.2
+      COMMONS_REVISION: 6e65371a2e11bef700e6493987aa81f9f73e95a3
+      PYTHONDONTWRITEBYTECODE: "1"
+    steps:
+      - name: Check out SDK revision
+        uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7
+        with:
+          path: operator-go
+          persist-credentials: false
+
+      - name: Check out fixed platform restarter
+        uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7
+        with:
+          repository: zncdatadev/commons-operator
+          ref: ${{ env.COMMONS_REVISION }}
+          path: commons-operator
+          persist-credentials: false
+
+      # The pinned commons checkout requires Go 1.25.8, satisfying the SDK's
+      # earlier 1.25.3 minimum without an implicit toolchain switch in that build.
+      - name: Set up Go for both modules and restarter
+        uses: actions/setup-go@b7ad1dad31e06c5925ef5d2fc7ad053ef454303e # v7
+        with:
+          go-version-file: commons-operator/go.mod
+          cache-dependency-path: |
+            operator-go/go.sum
+            operator-go/examples/trino-operator/go.sum
+            commons-operator/go.sum
+
+      # Docker and Python are part of the official ubuntu-24.04-arm image.
+      # Start/check that daemon explicitly; record its actual version as evidence.
+      - name: Check native ARM64 Docker and Python
+        run: |
+          test "$(uname -m)" = aarch64
+          sudo systemctl start docker
+          docker version
+          case "$(docker info --format '{{.Architecture}}')" in
+            aarch64|arm64) ;;
+            *) exit 1 ;;
+          esac
+          python3 --version
+          test "$(git -C ../commons-operator rev-parse HEAD)" = "$COMMONS_REVISION"
+
+      - name: Install pinned kind and kubectl with checksums
+        run: |
+          tool_dir="$RUNNER_TEMP/framework-tools"
+          mkdir -p "$tool_dir"
+          kind_base="https://github.com/kubernetes-sigs/kind/releases/download/$KIND_VERSION"
+          curl --fail --silent --show-error --location --retry 3 \
+            "$kind_base/kind-linux-arm64" -o "$tool_dir/kind-linux-arm64"
+          curl --fail --silent --show-error --location --retry 3 \
+            "$kind_base/kind-linux-arm64.sha256sum" -o "$tool_dir/kind-linux-arm64.sha256sum"
+          (cd "$tool_dir" && sha256sum --check kind-linux-arm64.sha256sum)
+          install -m 0755 "$tool_dir/kind-linux-arm64" "$tool_dir/kind"
+          kubectl_base="https://dl.k8s.io/release/$KUBECTL_VERSION/bin/linux/arm64"
+          curl --fail --silent --show-error --location --retry 3 \
+            "$kubectl_base/kubectl" -o "$tool_dir/kubectl"
+          curl --fail --silent --show-error --location --retry 3 \
+            "$kubectl_base/kubectl.sha256" -o "$tool_dir/kubectl.sha256"
+          (cd "$tool_dir" && printf '%s  kubectl\n' "$(cat kubectl.sha256)" | sha256sum --check)
+          chmod 0755 "$tool_dir/kubectl"
+          printf '%s\n' "$tool_dir" >> "$GITHUB_PATH"
+          make kustomize
+          printf '%s\n' "$PWD/bin" >> "$GITHUB_PATH"
+
+      - name: Verify generation, unit tests and lint before runtime
+        run: |
+          make verify-generate
+          make test ENVTEST_K8S_VERSION=1.35.0
+          make lint
+          assets="$(bin/setup-envtest use 1.35.0 --bin-dir "$PWD/bin" -p path)"
+          make -C examples/trino-operator test KUBEBUILDER_ASSETS="$assets"
+          make -C examples/trino-operator lint
+          git diff --exit-code
+
+      - name: Record installed tool versions
+        run: |
+          mkdir -p "$RUNNER_TEMP/framework-tool-evidence"
+          docker version > "$RUNNER_TEMP/framework-tool-evidence/docker.txt"
+          kind version > "$RUNNER_TEMP/framework-tool-evidence/kind.txt"
+          kubectl version --client -o json > "$RUNNER_TEMP/framework-tool-evidence/kubectl.json"
+          kustomize version > "$RUNNER_TEMP/framework-tool-evidence/kustomize.txt"
+          go version > "$RUNNER_TEMP/framework-tool-evidence/go.txt"
+          python3 --version > "$RUNNER_TEMP/framework-tool-evidence/python.txt"
+
+      # SIGTERM reaches the harness before the job timeout, leaving four minutes
+      # for its process/image/cluster cleanup and separate cleanup verdict.
+      - name: Run formal deployment, Trino and Retain acceptance
+        run: |
+          timeout --signal=TERM --kill-after=240s 65m \
+            python3 -u -B hack/framework-e2e/run.py \
+              --commons "$GITHUB_WORKSPACE/commons-operator" \
+              --output "$RUNNER_TEMP/framework-runtime" \
+            2>&1 | tee "$RUNNER_TEMP/framework-runtime.log"
+
+      - name: Upload reviewable evidence without kubeconfig or binaries
+        if: always()
+        uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7
+        with:
+          name: framework-runtime-arm64-${{ github.run_id }}-${{ github.run_attempt }}
+          if-no-files-found: warn
+          retention-days: 14
+          path: |
+            ${{ runner.temp }}/framework-tool-evidence/*.txt
+            ${{ runner.temp }}/framework-tool-evidence/*.json
+            ${{ runner.temp }}/framework-runtime.log
+            ${{ runner.temp }}/framework-runtime/**/*.json
+            ${{ runner.temp }}/framework-runtime/**/*.log
+            ${{ runner.temp }}/framework-runtime/**/*.txt
+            !${{ runner.temp }}/framework-runtime/**/*kubeconfig*

.github/workflows/lint.yml

@@ -32,11 +32,7 @@ jobs:
         with:
           version: ${{ steps.golangci.outputs.version }}
 
-  # examples/trino-operator is a separate Go module with its own .golangci.yml and its own pinned
-  # golangci-lint version, and the job above lints the root module only — so the reference
-  # implementation downstream operators copy was never linted at all. Driven through the module's
-  # own Makefile so its pin stays the single source of truth (it currently differs from the root's,
-  # which is fine: they are independent modules).
+  # The example is a separate module/config, using the root-pinned linter binary.
   examples-lint:
     name: Go Lint (examples/trino-operator)
     runs-on: ubuntu-latest
@@ -51,6 +47,7 @@ jobs:
 
       - name: Run linter
         run: |
+          make golangci-lint
           make -C examples/trino-operator lint
 
   md-lint:

.github/workflows/test.yml

@@ -81,12 +81,9 @@ jobs:
         run: |
           make verify-generate
 
-  # examples/trino-operator is a separate Go module, so the root module's jobs never compiled it.
-  # It is the reference implementation downstream operators are told to copy, and it was green
-  # purely because nothing ran it. Its own Makefile pins its own tool versions, so it is driven
-  # through that rather than reimplementing the setup here.
-  #
-  # The e2e suite still does not run: it needs a Kind cluster (see the module's `test-e2e` target).
+  # The formal Trino reference remains a separate module. Its real API-server
+  # sample test requires explicit root envtest assets; runtime delivery has a
+  # separate manually dispatched ARM64 workflow.
   examples-test:
     name: Go Test (examples/trino-operator)
     permissions:
@@ -103,4 +100,7 @@ jobs:
 
       - name: Running Tests
         run: |
-          make -C examples/trino-operator test
+          make setup-envtest ENVTEST_K8S_VERSION=1.35.0
+          assets="$(bin/setup-envtest use 1.35.0 --bin-dir "$PWD/bin" -p path)"
+          make -C examples/trino-operator test KUBEBUILDER_ASSETS="$assets"
+          make -C examples/trino-operator build

.gitignore

@@ -201,3 +201,6 @@ _bmad*
 .claude/worktrees
 .serena
 .worktree/
+
+# Local discussion, iteration notes and runtime evidence (never commit)
+/.local/

AGENTS.md

@@ -3,7 +3,7 @@
 ## Project Overview
 `operator-go` is a Golang SDK/framework for building Kubernetes operators. It provides a reusable reconciliation framework, CRDs, and utilities for creating product-specific operators.
 
-**Key Features:**
+**Key features of the existing GenericReconciler SDK:**
 - **GenericReconciler**: Template Method Pattern-based reconciliation framework
 - **Extension System**: Hook-based customization at cluster/role/role-group levels, with per-product registries
 - **Resource Builders**: Fluent builders for StatefulSet, Service, ConfigMap, PDB, RBAC, ServiceAccount
@@ -22,19 +22,40 @@
 
 > **IMPORTANT**: The `docs/` directory contains architecture documents that are the **authoritative source of design constraints** for this project. All implementations — including the SDK itself and any operators built with it — **must follow** the design defined in these documents. When code and documentation conflict, the documentation takes precedence. Consult these docs before making design decisions.
 
-> **Scope of that rule.** `docs/architecture.md` is authoritative about **design intent**: a
+> **Scope of that rule.** `docs/architecture.md` is the entry point for **design intent**. Its
+> applicability section routes new-framework implementation to its `framework-design` section,
+> while the separately scoped existing numbered sections apply to the GenericReconciler SDK. In the applicable scope, a
 > conflict means the code should change, not that the doc should be quietly relaxed. The
 > `AGENTS.md` files (this one and the per-package ones) are the opposite: they describe the API and
 > behavior that **exist today**, and must be corrected whenever the code changes. Never treat a
 > statement in an `AGENTS.md` as a requirement the code has yet to meet — anything aspirational
-> belongs in `docs/architecture.md` and must be explicitly labelled as such.
+> belongs in the applicable architecture specification and must be explicitly labelled as such.
+
+The new framework exposes contracts, generated input and registration in
+`pkg/framework`; see [package instructions](pkg/framework/AGENTS.md).
+Its pure build pipeline and controller are internal; the Trino reference uses
+the public contracts and generated registration as its actual executable.
+See [Trino instructions](examples/trino-operator/AGENTS.md) and the
+[delivery guide](hack/framework-e2e/README.md).
+The numbered SDK concepts below continue to describe the existing GenericReconciler API.
+
+### Local working records
+
+Store discussion notes, research, iteration plans, implementation progress, experimental
+prototypes and runtime evidence under `.local/engineering-notes/`, which is Git-ignored.
+The current discussion archive is `.local/engineering-notes/framework-redesign/README.md`.
+Do not add these records to Git or place them under `docs/`; do not use `git add -f`
+to bypass the ignore rule. Keep only maintained specifications, developer guidance and
+examples in `docs/`. Reusable tests and verification tools stay with source/test tooling;
+their generated reports belong in the local directory. Public docs and build/test paths
+must not depend on local records being present.
 
 ### Documentation Structure
 
 | File | Description |
 |------|-------------|
-| `docs/architecture.md` | **Core Technical Architecture** — design philosophy, layered architecture, core module specifications, design patterns, key problem solutions. This is the primary reference for all SDK design decisions. |
-| `docs/security.md` | **Security Architecture** — application security (SecretClass, CSI, AutoTLS, Kerberos) and infrastructure security (RBAC, ServiceAccounts, Pod security) |
+| `docs/architecture.md` | **Architecture specification** — product-description framework contracts and domain boundaries, followed by separately scoped GenericReconciler SDK sections. |
+| `docs/security.md` | **Security specification** — framework authentication, secrets and explicit data-operation authorization; existing SDK security sections are scoped separately. |
 | `docs/DOC_CHANGELOG.md` | Changelog tracking all documentation updates |
 | `docs/examples/` | CRD example YAMLs demonstrating the SDK's data model |
 

Dockerfile.dataops

@@ -0,0 +1,5 @@
+# Build context: bin/dataops-image, produced by make dataops-image.
+FROM scratch
+COPY dataops /dataops
+USER 65532:65532
+ENTRYPOINT ["/dataops"]

Makefile

@@ -21,18 +21,29 @@ help: ## Display this help.
 
 ##@ Development
 
+MATERIALIZER_IMG ?= quay.io/zncdatadev/operator-go-materializer:0.0.0-dev
+MATERIALIZER_ARCH ?= $(shell go env GOARCH)
+MATERIALIZER_CONTEXT = $(LOCALBIN)/materializer-$(MATERIALIZER_ARCH)
+
+.PHONY: materializer-build
+materializer-build: ## Compile the formal materializer as a static Linux binary.
+	mkdir -p "$(MATERIALIZER_CONTEXT)"
+	CGO_ENABLED=0 GOOS=linux GOARCH=$(MATERIALIZER_ARCH) go build -mod=readonly -trimpath -buildvcs=false -o "$(MATERIALIZER_CONTEXT)/materialize" ./cmd/materialize
+
+.PHONY: materializer-image
+materializer-image: materializer-build ## Build the materializer image; no registry push.
+	docker build --network=none --platform linux/$(MATERIALIZER_ARCH) --build-arg SOURCE_REVISION="$$(git rev-parse HEAD)" -f cmd/materialize/Dockerfile -t "$(MATERIALIZER_IMG)" "$(MATERIALIZER_CONTEXT)"
+
 .PHONY: generate
 generate: controller-gen ## Generate code containing DeepCopy, DeepCopyInto, and DeepCopyObject method implementations.
 	$(CONTROLLER_GEN) object:headerFile="hack/boilerplate.go.txt" paths="./pkg/..."
 
 .PHONY: manifests
-manifests: controller-gen ## Generate the CRDs backing the test mock cluster resources.
-# This SDK ships API types, not CRDs — a product operator generates its own from the types it
-# embeds. The only CRDs generated here are the ones envtest installs for pkg/testutil's mock
-# cluster resources, and they are generated rather than hand-written on purpose: a hand-written
-# CRD drifts from the Go types, and the schema-free version this replaced meant the API server
-# performed no defaulting, validation or pruning in ANY test in the repository.
+manifests: controller-gen ## Generate test cluster and independent framework data CRDs.
+# Products generate their own cluster CRDs. The framework also owns the independent
+# data identity/operation protocol and the mock cluster CRDs used by envtest.
 	$(CONTROLLER_GEN) crd paths="./pkg/testutil/..." output:crd:artifacts:config=config/crd/bases
+	$(CONTROLLER_GEN) crd paths="./pkg/framework/dataops/..." output:crd:artifacts:config=config/framework-data/bases
 
 .PHONY: fmt
 fmt: ## Run go fmt against code.
@@ -59,23 +70,12 @@ verify-generate: generate manifests ## Fail if the committed generated files are
 # Scoped to the paths generation writes, so the target stays usable with unrelated work in progress
 # — a check that fails on any dirty file is a check nobody runs locally.
 #
-# examples/trino-operator is a separate module and is checked too: it embeds the commons API types,
-# so a change to pkg/apis leaves its CRD stale, and it is the reference implementation downstream
-# operators copy. Its own Makefile pins its own controller-gen, so it is driven through that.
-#
-# '*/config/rbac/*' is in the pathspec because that module's generated ClusterRole is the canonical
-# operator-side permission set docs/security.md §3.3 points adopters at. Without it, an edited
-# +kubebuilder:rbac marker whose regenerated role.yaml was never committed passed CI.
-#
-# The trailing /* is load-bearing, and its absence is why the sibling '*/config/crd/bases' entry had
-# been inert since it was written. A git pathspec containing a wildcard is wildmatched against the
-# FULL path with no directory-prefix expansion, so '*/config/rbac' matches a path that IS that
-# directory and never a file inside it — the guard passed unconditionally. Verified by dirtying
-# examples/trino-operator/config/rbac/role.yaml: plain `git status` shows it, the old pathspec
-# reported nothing, the new one reports it. The literal `config/crd/bases` (no wildcard) always
-# worked, which is why only the root module was ever actually covered.
-	$(MAKE) -C examples/trino-operator generate manifests
-	@drift="$$(git status --porcelain -- '*zz_generated*.go' '*/config/crd/bases/*' config/crd/bases '*/config/rbac/*')"; \
+# The formal Trino example is a separate module. Its own generator checks the
+# committed input, CRD and registration companion against the product definition.
+# It does not use controller-gen or generate its deployment RBAC. Keep the scoped
+# status guard for root generated files and committed example delivery inputs.
+	$(MAKE) -C examples/trino-operator verify-generate
+	@drift="$$(git status --porcelain -- '*zz_generated*.go' '*/config/crd/bases/*' config/crd/bases config/framework-data/bases '*/config/rbac/*')"; \
 	if [ -n "$$drift" ]; then \
 		echo "Generated files are out of date. Run 'make generate manifests' and commit the result:"; \
 		echo "$$drift"; \
@@ -171,3 +171,13 @@ $(ENVTEST): $(LOCALBIN)
 golangci-lint: $(GOLANGCI_LINT) ## Download golangci-lint locally if necessary.
 $(GOLANGCI_LINT): $(LOCALBIN)
 	$(call go-install-tool,$(GOLANGCI_LINT),github.com/golangci/golangci-lint/v2/cmd/golangci-lint,$(GOLANGCI_LINT_VERSION))
+
+DATAOPS_IMAGE ?= operator-go-dataops:dev
+DATAOPS_ARCH ?= $(shell go env GOARCH)
+.PHONY: dataops-build dataops-image
+dataops-build: ## Build the independently deployed explicit-data executor.
+	mkdir -p bin/dataops-image
+	CGO_ENABLED=0 GOOS=linux GOARCH=$(DATAOPS_ARCH) go build -mod=readonly -trimpath -buildvcs=false -o bin/dataops-image/dataops ./cmd/dataops
+
+dataops-image: dataops-build ## Package the data executor separately from product operators.
+	docker build --platform linux/$(DATAOPS_ARCH) -f Dockerfile.dataops -t $(DATAOPS_IMAGE) bin/dataops-image

README.md

@@ -11,6 +11,12 @@ English | [简体中文](./README_zh-CN.md)
 
 A Golang SDK/framework for building Kubernetes operators. Built on [controller-runtime](https://github.com/kubernetes-sigs/controller-runtime), it provides a reusable reconciliation framework, CRDs, and utilities for creating product-specific operators.
 
+## New framework reference
+
+The new `pkg/framework` API lets a product declare its configuration and runtime while the SDK owns folding, overrides, resource assembly and reconciliation. Start with the [Trino reference operator](examples/trino-operator/README.md), [core specification](docs/architecture.md#framework-design) and [delivery/validation guide](hack/framework-e2e/README.md). Use the reviewed source revision for these packages; their presence here does not assert that `@latest` or a public image release already includes them.
+
+The remaining overview and quick start below describe the existing GenericReconciler API. They are not the authoring path used by the new Trino reference.
+
 ## Overview
 
 **operator-go** is designed to work seamlessly with [Kubebuilder](https://book.kubebuilder.io/), the standard framework for building Kubernetes APIs. We recommend following the [Kubebuilder documentation](https://book.kubebuilder.io/quick-start.html) to scaffold your operator project, then integrate operator-go to leverage its powerful reconciliation framework.
@@ -120,10 +126,10 @@ func (r *TrinoClusterReconciler) Reconcile(ctx context.Context, req ctrl.Request
 
 A complete example operator is available in [`examples/trino-operator/`](./examples/trino-operator/), demonstrating:
 
-- CRD definition with `ClusterInterface` implementation
-- RoleGroupHandler for coordinator and worker roles
-- Extension registration for custom logic
-- Webhook setup for validation and defaulting
+- Typed product configuration and generated presence-preserving API/CRD
+- ProductDefinition runtime generation for coordinator and worker roles
+- Generated registration through the public framework/operator API
+- Platform dependencies, logging, typed lifecycle and retained-storage consumers; runtime evidence is recorded separately
 
 ## Development