diff --git a/docs/developer-manual/develop-guideline.md b/docs/developer-manual/develop-guideline.md index 5cdd6f85..e684e7fe 100644 --- a/docs/developer-manual/develop-guideline.md +++ b/docs/developer-manual/develop-guideline.md @@ -1,4 +1,175 @@ +--- +title: Development Guideline +--- -# 开发指南 +This page covers developing Kubedoop itself — the operators and the framework they share. For +writing documentation see [Document Writing Guidelines](./document-guideline.md), and for the fork +and pull request mechanics see [First Contribution](./first-commiter.md). -TODO +## What you would be working on + +Kubedoop is not one repository. Each piece lives under +[github.com/zncdatadev](https://github.com/zncdatadev): + +| Repository | Contains | +|------------|----------| +| `operator-go` | The Go framework every operator builds on: shared CRD types, the reconciler, resource builders | +| `-operator` | One operator per product — `zookeeper-operator`, `hdfs-operator`, `trino-operator`, and so on | +| `commons-operator`, `listener-operator`, `secret-operator` | The built-in operators every product cluster depends on | +| `containers` | Product container images | +| `kubedoop-helm-charts` | Published Helm charts | +| `docs` | This site | + +Each operator is an independent Go module that depends on `operator-go`. A change to shared +behaviour usually belongs in the framework; a change to one product's rendering belongs in that +product's operator. + +## Prerequisites + +| Tool | Why | +|------|-----| +| Go | Building and testing. The required version is in each repository's `go.mod` | +| Docker or Podman | Building operator images | +| kubectl | Talking to a cluster | +| [kind](https://kind.sigs.k8s.io/) | The local cluster end-to-end tests run against | +| Helm | Installing operator dependencies during e2e | + +Everything else — `controller-gen`, `kustomize`, `setup-envtest`, `golangci-lint`, `chainsaw` — is +downloaded into the repository's `bin/` directory by the Makefile on first use. Do not install those +globally; a mismatched version is a source of failures that do not reproduce in CI. + +Kubernetes 1.29 is the supported floor. + +## Repository layout + +Operators follow the standard [Kubebuilder](https://book.kubebuilder.io/) layout, with a few +Kubedoop additions: + +```text +api/v1alpha1/ # CRD Go types — the API surface +internal/ # Reconcilers, builders, product-specific logic +cmd/ # Manager entrypoint +config/ # Kustomize manifests, including generated CRDs +deploy/helm/ # The operator's Helm chart +test/e2e/ # Chainsaw end-to-end suites +examples/ # Example custom resources +Makefile # Every task below +PROJECT # Kubebuilder project metadata +``` + +`make help` lists the targets in any operator repository. + +## The development loop + +```bash +make manifests generate # regenerate CRDs and deepcopy code after API changes +make fmt vet # format and vet +make lint # golangci-lint +make test # unit tests +make build # build the manager binary +make run # run the controller against your current kubecontext +``` + +`make test` depends on `manifests generate fmt vet`, so it regenerates before running. If that +produces a diff, commit it — generated files are checked in. + +## Changing the API + +API types live in `api/v1alpha1/`. After editing them: + +```bash +make manifests generate +``` + +This regenerates the CRD YAML under `config/` and the `zz_generated.deepcopy.go` files. Both are +committed, so a change to a type with no accompanying regeneration will show up as a diff in CI. + +The Helm chart carries its own copy of the CRDs and the operator's RBAC rules, and they do not +update themselves: + +```bash +make helm-crd-sync # copy generated CRDs into deploy/helm/ +make helm-rbac-sync # copy the generated ClusterRole rules into deploy/helm/ +``` + +Forgetting these is the usual cause of a chart that installs an operator against a stale CRD. + +### Adding fields + +Prefer the shared structures in `operator-go`'s `pkg/apis/commons/v1alpha1` — resources, logging, +affinity, PodDisruptionBudget — over defining a product-local equivalent. A field that ends up +copied into three operators is a sign it belongs in the framework instead. + +Be careful with `+kubebuilder:default` on fields inside `config`. That block is folded from role to +role group, and structural defaulting fills a leaf as soon as its enclosing object exists — so a +default there makes "unset" indistinguishable from "explicitly this value", and the role's setting +can never win. Several fields in the framework carry comments explaining exactly this; defaults for +folded fields belong at consumption time. + +## Testing + +### Unit tests + +```bash +make test +``` + +These run against [envtest](https://book.kubebuilder.io/reference/envtest.html) — a real API server +and etcd, no scheduler or controllers. Enough to assert what the operator *renders*: that a given +CR produces the StatefulSet, ConfigMap and Service you expect. + +### End-to-end tests + +End-to-end tests use [Chainsaw](https://kyverno.github.io/chainsaw/) against a kind cluster. Cases +live in `test/e2e///`, each a directory of YAML steps and assertions. + +```bash +make setup-chainsaw-cluster # create the kind cluster, install operator dependencies +make setup-chainsaw-e2e # build the image, load it, deploy the operator +make chainsaw-e2e # run the suites +make cleanup-chainsaw-e2e # undeploy +``` + +`setup-chainsaw-cluster` installs the operators the product depends on — `commons-operator`, +`listener-operator` and `secret-operator` — from the published charts, because almost nothing works +without them: secrets, listeners and pod enrichment are all provided by those three. + +Write an end-to-end case for anything that only shows up on a live cluster: rendered configuration +a unit test would have to duplicate, rolling behaviour, discovery between two products. Keep +assertions on the resources rather than on log output. + +## Working with operator-go + +Operators depend on a released `operator-go`. When a change needs both the framework and an operator +at once, point the module at your local checkout while developing: + +```text +replace github.com/zncdatadev/operator-go => ../operator-go +``` + +Remove the `replace` before opening the pull request — it cannot be merged. Land the framework +change first, then bump the operator to the released version. + +## Before opening a pull request + +Run the same gates CI does, from the operator's repository: + +```bash +make manifests generate # and commit any resulting diff +make fmt vet lint +make test +make helm-crd-sync helm-rbac-sync # if the API changed +``` + +Then the end-to-end suite for anything touching rendering or reconcile behaviour. + +Follow the commit and branch conventions in [First Contribution](./first-commiter.md), and describe +in the pull request what you actually ran. A checklist box ticked without running the command is +worse than an honest note about what was skipped. + +## Related + +- [Collaboration Guide](./collaboration.md) +- [First Contribution](./first-commiter.md) +- [Document Writing Guidelines](./document-guideline.md) +- [Project Status](./project-status.md) diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/developer-manual/develop-guideline.md b/i18n/zh/docusaurus-plugin-content-docs/current/developer-manual/develop-guideline.md index 5cdd6f85..8d2b4efa 100644 --- a/i18n/zh/docusaurus-plugin-content-docs/current/developer-manual/develop-guideline.md +++ b/i18n/zh/docusaurus-plugin-content-docs/current/developer-manual/develop-guideline.md @@ -1,4 +1,169 @@ +--- +title: 开发指南 +--- -# 开发指南 +本页讲的是开发 Kubedoop 本身——各个 Operator 以及它们共用的框架。撰写文档请看 +[文档编写规范](./document-guideline.md),fork 与 PR 的具体流程请看 +[第一次贡献](./first-commiter.md)。 -TODO +## 你会改动哪些代码 + +Kubedoop 不是单个仓库,各部分都在 +[github.com/zncdatadev](https://github.com/zncdatadev) 下: + +| 仓库 | 内容 | +|------|------| +| `operator-go` | 所有 Operator 共用的 Go 框架:共享 CRD 类型、调谐器、资源构建器 | +| `-operator` | 每个产品一个 Operator——`zookeeper-operator`、`hdfs-operator`、`trino-operator` 等 | +| `commons-operator`、`listener-operator`、`secret-operator` | 所有产品集群都依赖的内置 Operator | +| `containers` | 产品容器镜像 | +| `kubedoop-helm-charts` | 已发布的 Helm chart | +| `docs` | 本站点 | + +每个 Operator 都是独立的 Go module,依赖 `operator-go`。改动共性行为通常应该落在框架里, +只影响某个产品渲染结果的改动才放进该产品的 Operator。 + +## 前置条件 + +| 工具 | 用途 | +|------|------| +| Go | 构建与测试。所需版本见各仓库的 `go.mod` | +| Docker 或 Podman | 构建 Operator 镜像 | +| kubectl | 与集群交互 | +| [kind](https://kind.sigs.k8s.io/) | 端到端测试所用的本地集群 | +| Helm | e2e 过程中安装 Operator 依赖 | + +其余工具——`controller-gen`、`kustomize`、`setup-envtest`、`golangci-lint`、`chainsaw`—— +由 Makefile 在首次使用时下载到仓库的 `bin/` 目录。不要全局安装它们, +版本不一致会导致本地失败而 CI 无法复现(反之亦然)。 + +Kubernetes 支持的最低版本是 1.29。 + +## 仓库结构 + +Operator 遵循标准 [Kubebuilder](https://book.kubebuilder.io/) 布局,另有几处 Kubedoop 特有的目录: + +```text +api/v1alpha1/ # CRD Go 类型——对外 API 面 +internal/ # 调谐器、构建器、产品特有逻辑 +cmd/ # manager 入口 +config/ # Kustomize 清单,含生成的 CRD +deploy/helm/ # 该 Operator 的 Helm chart +test/e2e/ # Chainsaw 端到端用例 +examples/ # 示例自定义资源 +Makefile # 下文所有任务 +PROJECT # Kubebuilder 项目元数据 +``` + +在任意 Operator 仓库执行 `make help` 可列出全部目标。 + +## 开发循环 + +```bash +make manifests generate # 改完 API 后重新生成 CRD 与 deepcopy 代码 +make fmt vet # 格式化与静态检查 +make lint # golangci-lint +make test # 单元测试 +make build # 构建 manager 二进制 +make run # 用当前 kubecontext 在本机运行控制器 +``` + +`make test` 依赖 `manifests generate fmt vet`,因此会先重新生成。如果生成产生了 diff,请提交它 +——生成物是纳入版本控制的。 + +## 修改 API + +API 类型位于 `api/v1alpha1/`。修改后执行: + +```bash +make manifests generate +``` + +这会重新生成 `config/` 下的 CRD YAML 和各处 `zz_generated.deepcopy.go`。两者都要提交, +所以改了类型却没重新生成,会在 CI 里表现为 diff。 + +Helm chart 自己保存了一份 CRD 和 Operator 的 RBAC 规则,它们**不会自动更新**: + +```bash +make helm-crd-sync # 把生成的 CRD 同步进 deploy/helm/ +make helm-rbac-sync # 把生成的 ClusterRole 规则同步进 deploy/helm/ +``` + +漏掉这两步,通常就是"chart 安装出来的 Operator 配着一份过期 CRD"的原因。 + +### 新增字段 + +优先复用 `operator-go` 的 `pkg/apis/commons/v1alpha1` 中已有的结构——资源、日志、亲和性、 +PodDisruptionBudget——而不是在产品侧另造一个等价物。一个字段如果被抄进了三个 Operator, +就说明它本该属于框架。 + +给 `config` 内部的字段加 `+kubebuilder:default` 要格外小心。该块会从角色折叠到角色组, +而结构化默认值会在其**外层对象**一存在时就填充叶子字段——于是"未设置"和"显式设成该值" +变得无法区分,角色级的取值就永远赢不了。框架里有若干字段的注释专门解释了这一点; +折叠字段的默认值应当在消费时才施加。 + +## 测试 + +### 单元测试 + +```bash +make test +``` + +它们运行在 [envtest](https://book.kubebuilder.io/reference/envtest.html) 之上—— +一个真实的 API server 加 etcd,但没有调度器和内置控制器。这足以断言 Operator **渲染出了什么**: +给定一个 CR,是否产出了预期的 StatefulSet、ConfigMap 和 Service。 + +### 端到端测试 + +端到端测试使用 [Chainsaw](https://kyverno.github.io/chainsaw/),跑在 kind 集群上。 +用例位于 `test/e2e///`,每个用例是一个装着 YAML 步骤与断言的目录。 + +```bash +make setup-chainsaw-cluster # 创建 kind 集群,安装 Operator 依赖 +make setup-chainsaw-e2e # 构建镜像、加载、部署 Operator +make chainsaw-e2e # 运行用例 +make cleanup-chainsaw-e2e # 卸载 +``` + +`setup-chainsaw-cluster` 会从已发布的 chart 安装产品所依赖的 Operator——`commons-operator`、 +`listener-operator` 和 `secret-operator`——因为缺了它们几乎什么都跑不起来: +secret、listener 和 Pod 信息补全都由这三个提供。 + +凡是只有在真实集群上才暴露的行为,都应该写端到端用例:单元测试需要重复实现一遍才能验证的渲染结果、 +滚动更新行为、两个产品之间的服务发现。断言请针对资源本身,而不是日志输出。 + +## 与 operator-go 协同开发 + +Operator 依赖的是已发布的 `operator-go`。当一处改动同时需要框架和 Operator 时, +开发期间可以把 module 指向本地检出: + +```text +replace github.com/zncdatadev/operator-go => ../operator-go +``` + +提 PR 之前务必移除这个 `replace`,带着它无法合并。正确顺序是先让框架的改动合入, +再把 Operator 升到已发布的版本。 + +## 提 PR 之前 + +在 Operator 仓库里跑一遍 CI 会跑的那些门禁: + +```bash +make manifests generate # 并提交由此产生的 diff +make fmt vet lint +make test +make helm-crd-sync helm-rbac-sync # 如果改了 API +``` + +涉及渲染或调谐行为的改动,再跑一遍端到端用例。 + +提交与分支命名请遵循[第一次贡献](./first-commiter.md)中的约定,并在 PR 里如实写明你实际跑了什么。 +勾了却没真跑的检查项,比诚实说明"这一项跳过了"更糟。 + +## 相关内容 + +- [协作指南](./collaboration.md) +- [第一次贡献](./first-commiter.md) +- [文档编写规范](./document-guideline.md) +- [项目状态](./project-status.md)