diff --git a/docs/developer-manual/develop-guideline.md b/docs/developer-manual/develop-guideline.md index e684e7fe..b7fa2f5d 100644 --- a/docs/developer-manual/develop-guideline.md +++ b/docs/developer-manual/develop-guideline.md @@ -38,7 +38,10 @@ Everything else — `controller-gen`, `kustomize`, `setup-envtest`, `golangci-li downloaded into the repository's `bin/` directory by the Makefile on first use. Do not install those globally; a mismatched version is a source of failures that do not reproduce in CI. -Kubernetes 1.29 is the supported floor. +End-to-end tests run against the Kubernetes versions listed in +[Kubernetes](../user-manual/environment/kubernetes.md#supported-versions). Note that +`KIND_K8S_VERSION` in each Makefile is only the default for a local cluster and is older than what +CI exercises — CI overrides it with its own matrix. ## Repository layout diff --git a/docs/operators/spark-k8s-operator.md b/docs/operators/spark-k8s-operator.md index 66d703f2..451c2473 100644 --- a/docs/operators/spark-k8s-operator.md +++ b/docs/operators/spark-k8s-operator.md @@ -22,7 +22,7 @@ them. ## Prerequisites -- Kubernetes 1.29+ +- Kubernetes — see [supported versions](../user-manual/environment/kubernetes.md#supported-versions) - kubectl - Helm v3+ — see [Installation](../quick-start/installation.md) - An S3-compatible object store holding the event logs, reachable from the cluster diff --git a/docs/user-manual/environment/kubernetes.md b/docs/user-manual/environment/kubernetes.md index 8b25ad01..3f718edf 100644 --- a/docs/user-manual/environment/kubernetes.md +++ b/docs/user-manual/environment/kubernetes.md @@ -1 +1,126 @@ -# Kubernetes environment +--- +title: Kubernetes +--- + +Kubedoop runs on any conformant Kubernetes cluster. This page covers the versions it is tested +against and the handful of cluster capabilities the built-in Operators depend on. + +## Supported versions + +Every Operator's end-to-end suite runs against the same matrix on each pull request and each +release: + +| Kubernetes | Status | +|------------|--------| +| 1.35 | Tested | +| 1.34 | Tested | +| 1.33 | Tested | + +Other versions are not exercised by CI. Older releases may well work — nothing in the Operators +deliberately requires a recent API — but if you run one, you are the first to find out. + +The Helm charts declare no `kubeVersion` constraint, so Helm will not stop you installing on +anything. Treat the table above as the supported range rather than a limit the tooling enforces. + +## Cluster requirements + +### CSI + +Two of the built-in Operators are CSI drivers rather than ordinary controllers: + +| Operator | CSI driver | Provides | +|----------|------------|----------| +| `secret-operator` | `secrets.kubedoop.dev` | Delivers credentials and certificates into pods as volumes | +| `listener-operator` | `listeners.kubedoop.dev` | Exposes pods and reports the address back into the pod | + +Both register a `CSIDriver` object and run a node component as a DaemonSet, so the cluster must +allow CSI drivers and let those pods mount the kubelet directory. A managed cluster that forbids +privileged workloads outright will not be able to run them, and without them almost nothing else +works — every product cluster gets its secrets and its service exposure through these two. + +`secret-operator`'s driver is registered with `attachRequired: false`, `podInfoOnMount: true` and +both `Ephemeral` and `Persistent` volume lifecycle modes. + +### The kubelet directory + +The node components mount the kubelet's plugin directories from the host. The chart default is: + +```yaml +kubeletDir: /var/lib/kubelet +``` + +**Distributions that put the kubelet somewhere else need this overridden**, or the driver never +registers and every pod that wants a secret volume stays `Pending`. microk8s and k3s are the usual +cases: + +```bash +helm install secret-operator kubedoop/secret-operator -n operators \ + --set kubeletDir=/var/snap/microk8s/common/var/lib/kubelet +``` + +Check where your distribution keeps it before installing: + +```bash +ps aux | grep kubelet | grep -o '\-\-root-dir=[^ ]*' +``` + +If that prints nothing, the default is in use. + +### Storage + +Products that keep state — HDFS, Kafka, ZooKeeper — request PersistentVolumeClaims, so the cluster +needs a working `StorageClass`. Most managed clusters ship one; a bare kind cluster has +`standard` from the local-path provisioner. + +```bash +kubectl get storageclass +``` + +`listener-operator` additionally creates its own StorageClass, `listeners.listeners.kubedoop.dev`, +used by the ephemeral volumes that carry listener addresses. That one is installed for you. + +### RBAC + +The Operators create and manage StatefulSets, Services, ConfigMaps, PodDisruptionBudgets and their +own custom resources. Installing via Helm creates the ClusterRoles they need, so you need +permission to create cluster-scoped RBAC at install time. + +## Local clusters + +[kind](https://kind.sigs.k8s.io/) is what the project's own end-to-end tests use, and it is the +quickest way to try Kubedoop: + +```bash +kind create cluster --image kindest/node:v1.35.0 +``` + +minikube and k3d work too. On k3d, remember the kubelet directory note above. + +Size the cluster for what you deploy rather than for Kubedoop itself: the Operators are small, but a +three-node ZooKeeper ensemble plus HDFS plus a query engine will not fit in a default single-node +kind cluster without raising its resources. + +## Verifying a cluster + +Before installing, confirm the basics: + +```bash +kubectl version # server version within the supported range +kubectl get storageclass # at least one, ideally a default +kubectl auth can-i create clusterrole # needed for the Helm install +``` + +After installing the built-in Operators, check that the CSI drivers registered: + +```bash +kubectl get csidrivers +``` + +Both `secrets.kubedoop.dev` and `listeners.kubedoop.dev` should be listed. If they are missing, the +kubelet directory is the first thing to check. + +## Related + +- [Installation](../../quick-start/installation.md) +- [Service Discovery](../../core-concepts/connectivity/service-discovery.md) +- [Resource Management](../../core-concepts/resources/resource-manage.md) diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/developer-manual/develop-guideline.md b/i18n/zh/docusaurus-plugin-content-docs/current/developer-manual/develop-guideline.md index 8d2b4efa..cd305b8a 100644 --- a/i18n/zh/docusaurus-plugin-content-docs/current/developer-manual/develop-guideline.md +++ b/i18n/zh/docusaurus-plugin-content-docs/current/developer-manual/develop-guideline.md @@ -37,7 +37,9 @@ Kubedoop 不是单个仓库，各部分都在 由 Makefile 在首次使用时下载到仓库的 `bin/` 目录。不要全局安装它们， 版本不一致会导致本地失败而 CI 无法复现（反之亦然）。 -Kubernetes 支持的最低版本是 1.29。 +端到端测试运行在 [Kubernetes](../user-manual/environment/kubernetes.md#支持的版本) 页列出的版本上。 +注意各 Makefile 里的 `KIND_K8S_VERSION` 只是本地集群的默认值，比 CI 实际覆盖的版本旧—— +CI 会用自己的矩阵覆盖它。 ## 仓库结构 diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/operators/spark-k8s-operator.md b/i18n/zh/docusaurus-plugin-content-docs/current/operators/spark-k8s-operator.md index 84905982..280fba3d 100644 --- a/i18n/zh/docusaurus-plugin-content-docs/current/operators/spark-k8s-operator.md +++ b/i18n/zh/docusaurus-plugin-content-docs/current/operators/spark-k8s-operator.md @@ -19,7 +19,7 @@ title: Spark ## 前置条件 -- Kubernetes 1.29+ +- Kubernetes —— 参见[支持的版本](../user-manual/environment/kubernetes.md#支持的版本) - kubectl - Helm v3+ —— 参见[安装](../quick-start/installation.md) - 一个集群内可访问、存放事件日志的 S3 兼容对象存储 diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/user-manual/environment/kubernetes.md b/i18n/zh/docusaurus-plugin-content-docs/current/user-manual/environment/kubernetes.md index f84dee29..4db0ba7e 100644 --- a/i18n/zh/docusaurus-plugin-content-docs/current/user-manual/environment/kubernetes.md +++ b/i18n/zh/docusaurus-plugin-content-docs/current/user-manual/environment/kubernetes.md @@ -1 +1,120 @@ -# Kubernetes 环境 +--- +title: Kubernetes +--- + +Kubedoop 可以运行在任何符合规范的 Kubernetes 集群上。本页说明它经过测试的版本， +以及内置 Operator 依赖的少数几项集群能力。 + +## 支持的版本 + +每个 Operator 的端到端测试都会在每次 PR 和每次发版时跑同一套版本矩阵： + +| Kubernetes | 状态 | +|------------|------| +| 1.35 | 已测试 | +| 1.34 | 已测试 | +| 1.33 | 已测试 | + +其他版本 CI 不会覆盖。更老的版本很可能也能用——Operator 并没有刻意依赖新的 API—— +但真要跑，你就是第一个发现问题的人。 + +Helm chart 没有声明 `kubeVersion` 约束，因此 Helm 不会阻止你装在任何版本上。 +请把上表当作受支持的范围，而不是工具强制执行的限制。 + +## 集群要求 + +### CSI + +内置 Operator 中有两个是 CSI 驱动，而不是普通的控制器： + +| Operator | CSI 驱动 | 提供 | +|----------|----------|------| +| `secret-operator` | `secrets.kubedoop.dev` | 以卷的形式把凭据和证书送进 Pod | +| `listener-operator` | `listeners.kubedoop.dev` | 暴露 Pod，并把地址回写进 Pod | + +两者都会注册 `CSIDriver` 对象，并以 DaemonSet 形式运行节点组件，因此集群必须允许 CSI 驱动， +并允许这些 Pod 挂载 kubelet 目录。完全禁止特权工作负载的托管集群跑不了它们， +而缺了它们几乎什么都用不了——每个产品集群的密钥分发和服务暴露都要经过这两个 Operator。 + +`secret-operator` 的驱动注册参数为 `attachRequired: false`、`podInfoOnMount: true`， +并同时支持 `Ephemeral` 和 `Persistent` 两种卷生命周期模式。 + +### kubelet 目录 + +节点组件需要从宿主机挂载 kubelet 的插件目录。chart 默认值为： + +```yaml +kubeletDir: /var/lib/kubelet +``` + +**把 kubelet 放在别处的发行版必须覆盖这个值**，否则驱动永远注册不上， +所有需要密钥卷的 Pod 都会卡在 `Pending`。microk8s 和 k3s 是最常见的情况： + +```bash +helm install secret-operator kubedoop/secret-operator -n operators \ + --set kubeletDir=/var/snap/microk8s/common/var/lib/kubelet +``` + +安装前先确认你的发行版把它放在哪： + +```bash +ps aux | grep kubelet | grep -o '\-\-root-dir=[^ ]*' +``` + +没有输出说明用的是默认路径。 + +### 存储 + +有状态的产品——HDFS、Kafka、ZooKeeper——会申请 PersistentVolumeClaim， +因此集群需要一个可用的 `StorageClass`。大多数托管集群自带； +裸的 kind 集群带有 local-path provisioner 提供的 `standard`。 + +```bash +kubectl get storageclass +``` + +`listener-operator` 还会额外创建自己的 StorageClass `listeners.listeners.kubedoop.dev`， +供承载监听器地址的临时卷使用。这个不需要你操心。 + +### RBAC + +Operator 需要创建和管理 StatefulSet、Service、ConfigMap、PodDisruptionBudget 以及它们自己的 +自定义资源。通过 Helm 安装时会创建所需的 ClusterRole，因此安装者需要具备创建集群级 RBAC 的权限。 + +## 本地集群 + +[kind](https://kind.sigs.k8s.io/) 是项目自身端到端测试所使用的环境，也是试用 Kubedoop 最快的方式： + +```bash +kind create cluster --image kindest/node:v1.35.0 +``` + +minikube 和 k3d 同样可用。使用 k3d 时请注意上面关于 kubelet 目录的说明。 + +集群规格应当按你要部署的产品来定，而不是按 Operator 本身：Operator 很轻， +但三节点 ZooKeeper 加 HDFS 再加一个查询引擎，塞不进未调整资源的默认单节点 kind 集群。 + +## 检查集群 + +安装前先确认基础条件： + +```bash +kubectl version # 服务端版本在受支持范围内 +kubectl get storageclass # 至少有一个，最好有默认的 +kubectl auth can-i create clusterrole # Helm 安装需要 +``` + +安装完内置 Operator 后，确认 CSI 驱动已注册： + +```bash +kubectl get csidrivers +``` + +`secrets.kubedoop.dev` 和 `listeners.kubedoop.dev` 都应当出现在列表里。 +如果缺失，首先要排查的就是 kubelet 目录。 + +## 相关内容 + +- [安装](../../quick-start/installation.md) +- [服务发现](../../core-concepts/connectivity/service-discovery.md) +- [资源管理](../../core-concepts/resources/resource-manage.md)