diff --git a/docs/core-concepts/operations/pod-disruptions.md b/docs/core-concepts/operations/pod-disruptions.md index e69de29b..95c3e566 100644 --- a/docs/core-concepts/operations/pod-disruptions.md +++ b/docs/core-concepts/operations/pod-disruptions.md @@ -0,0 +1,86 @@ +--- +title: Pod Disruptions +--- + +A Kubernetes cluster moves pods around for reasons that have nothing to do with your workload: +draining a node for an upgrade, rebalancing, scaling down the node pool. These are *voluntary* +disruptions, and a +[PodDisruptionBudget](https://kubernetes.io/docs/concepts/workloads/pods/disruptions/) is how a +workload tells the cluster how many of its pods may be taken down at once. + +This matters most for quorum-based roles. Evicting two of three ZooKeeper servers at the same time +does not slow the ensemble down — it loses quorum and stops. + +Kubedoop writes a PodDisruptionBudget **per role**, using a default the operator picks from what it +knows about the product. You do not have to configure anything to be protected. + +## Configuration + +Each role accepts a `roleConfig.podDisruptionBudget` block: + +```yaml +apiVersion: zookeeper.kubedoop.dev/v1alpha1 +kind: ZookeeperCluster +metadata: + name: zookeeper +spec: + server: + roleConfig: + podDisruptionBudget: + enabled: true + maxUnavailable: 1 + roleGroups: + default: + replicas: 3 +``` + +| Field | Type | Default | Meaning | +|-------|------|---------|---------| +| `enabled` | boolean | `true` | Whether the operator writes a PDB for this role | +| `maxUnavailable` | integer | product-specific | How many pods of the role may be down at once | + +Both fields are optional, and the block itself is optional. + +### `enabled` defaults to true, including when you omit it + +Leaving `podDisruptionBudget` out entirely, and writing a block that only sets `maxUnavailable`, +both leave the PDB enabled. A role that mentions the budget just to raise `maxUnavailable` still +gets its PDB — you only lose it by writing `enabled: false` explicitly. + +### `maxUnavailable` defaults per product + +When you do not set it, the operator supplies a value based on what it knows about the product +rather than a blanket number, because the safe count differs by role: a three-node quorum tolerates +one loss, a pool of stateless workers tolerates far more. + +Set it yourself only when you know the product's tolerance better than the operator does for your +topology. + +## Supplying your own PDB + +`enabled: false` stops the operator from writing a PDB for the role, which is what you want when +you intend to manage one yourself: + +```yaml +spec: + server: + roleConfig: + podDisruptionBudget: + enabled: false +``` + +The operator then leaves the role alone, and any PodDisruptionBudget you create with your own +selector applies normally. Note that turning it off and *not* replacing it leaves the role with no +protection against voluntary eviction at all. + +## What a PDB does not cover + +A PodDisruptionBudget constrains voluntary disruptions only. A node that hard-crashes, a kernel +panic, or a pod killed for exceeding its memory limit are all involuntary — the budget has no say. +Spreading a role across failure domains is the tool for that; see +[Pod placement](./pod-placement.md). + +## Related + +- [Pod placement](./pod-placement.md) +- [Roles and role groups](../common-configuration-mechanisms/roles-and-role-groups.md) diff --git a/docs/core-concepts/operations/pod-placement.md b/docs/core-concepts/operations/pod-placement.md index e69de29b..49f1601a 100644 --- a/docs/core-concepts/operations/pod-placement.md +++ b/docs/core-concepts/operations/pod-placement.md @@ -0,0 +1,138 @@ +--- +title: Pod Placement +--- + +Where a role's pods land decides how much a single failure costs. Three ZooKeeper servers on one +node are three pods that disappear together. Kubedoop products therefore ship a default +[affinity](https://kubernetes.io/docs/concepts/scheduling-eviction/assign-pod-node/) that spreads a +role across nodes, and let you replace it when your topology calls for something else. + +## The default is already there + +Every product declares affinity defaults for its own roles — typically a `podAntiAffinity` that +keeps replicas of a role off the same node. You do not need to write anything to get it. + +Read the section below before overriding it, because overriding does not mean adding. + +## Configuring affinity + +Affinity is set under `config`, which exists at two levels: + +```yaml +apiVersion: zookeeper.kubedoop.dev/v1alpha1 +kind: ZookeeperCluster +metadata: + name: zookeeper +spec: + server: + config: + affinity: # role level: applies to every group of this role + nodeAffinity: + requiredDuringSchedulingIgnoredDuringExecution: + nodeSelectorTerms: + - matchExpressions: + - key: node-role.kubernetes.io/worker + operator: Exists + roleGroups: + default: + replicas: 3 + gpu: + replicas: 2 + config: + affinity: # role group level: applies to this group only + nodeAffinity: + requiredDuringSchedulingIgnoredDuringExecution: + nodeSelectorTerms: + - matchExpressions: + - key: accelerator + operator: In + values: ["nvidia"] +``` + +The value is a standard Kubernetes +[`Affinity`](https://kubernetes.io/docs/reference/kubernetes-api/workload-resources/pod-v1/#scheduling) +object and is passed through to the pod spec unchanged, so `nodeAffinity`, `podAffinity` and +`podAntiAffinity` all behave exactly as they do in a plain Deployment. + +## Affinity replaces, it does not merge + +There are three layers, from the bottom up: + +1. the product's role defaults +2. the role's `config.affinity` +3. the role group's `config.affinity` + +**Any layer that sets `affinity` replaces the layer beneath it entirely.** This follows the +Kubernetes rule and is what someone editing the field expects — but it has a consequence that is +easy to miss. + +Setting only `nodeAffinity` at the role group level, to pin a group to an instance type, discards +the product's `podAntiAffinity` along with it. The group is now pinned to the right nodes and no +longer spread across them, which is usually the opposite of what was intended. + +Nothing fails when this happens. The resource is valid, the pod spec is valid, and every status +condition stays green while the quorum quietly stops being spread. + +### How to tell it happened + +The operator emits a Warning event on the cluster resource: + +```text +Type Reason Message +Warning AffinityOverridden role "server" group "gpu": the role group's config replaces + config.affinity wholesale, discarding the podAntiAffinity declared + beneath it. config.affinity follows the Kubernetes rule and is not + merged per member; restate the discarded member alongside your own + to keep it +``` + +Check for it after changing affinity: + +```bash +kubectl get events --field-selector reason=AffinityOverridden +``` + +### Keeping what you replaced + +Restate the discarded member alongside your own: + +```yaml +config: + affinity: + nodeAffinity: + requiredDuringSchedulingIgnoredDuringExecution: + nodeSelectorTerms: + - matchExpressions: + - key: accelerator + operator: In + values: ["nvidia"] + podAntiAffinity: # restated, or it is lost + preferredDuringSchedulingIgnoredDuringExecution: + - weight: 70 + podAffinityTerm: + topologyKey: kubernetes.io/hostname + labelSelector: + matchLabels: + app.kubernetes.io/name: zookeeper + app.kubernetes.io/instance: zookeeper + app.kubernetes.io/component: server +``` + +## Choosing a topology key + +`topologyKey` decides what "spread apart" means: + +| Key | Spreads across | +|-----|----------------| +| `kubernetes.io/hostname` | Nodes — protects against a single node failing | +| `topology.kubernetes.io/zone` | Availability zones — protects against a zone failing | + +Zone-level spreading needs enough nodes per zone to satisfy it. With +`requiredDuringSchedulingIgnoredDuringExecution` and fewer zones than replicas, the surplus pods +stay `Pending` forever; `preferredDuringSchedulingIgnoredDuringExecution` degrades instead of +blocking, which is why product defaults favour it. + +## Related + +- [Pod disruptions](./pod-disruptions.md) +- [Roles and role groups](../common-configuration-mechanisms/roles-and-role-groups.md) diff --git a/docs/core-concepts/resources/s3.md b/docs/core-concepts/resources/s3.md index 6d45a35d..8c19465e 100644 --- a/docs/core-concepts/resources/s3.md +++ b/docs/core-concepts/resources/s3.md @@ -1,4 +1,171 @@ +--- +title: S3 +--- -# S3 +Many Kubedoop products read and write object storage: Hive keeps table data there, Trino queries it, +Spark checkpoints to it. Rather than repeating endpoint and credential settings in every product +resource, Kubedoop models S3 as two cluster resources that products point at. -TODO +## Two objects + +| Object | Describes | +|--------|-----------| +| `S3Connection` | Where the object store is and how to reach it: host, port, TLS, region, addressing style, credentials | +| `S3Bucket` | A bucket name, plus the connection that bucket lives on | + +Both belong to the `s3.kubedoop.dev/v1alpha1` API group. + +Which one a product asks for depends on how it addresses storage. A product handed a whole bucket +takes an `S3Bucket`; a product that decides bucket names itself — Hive Metastore, for instance — +takes an `S3Connection` directly. + +## Inline or reference, never both + +Wherever Kubedoop accepts S3 configuration it accepts the same pair of fields: `inline` to define +the object in place, or `reference` to name an existing object in the same namespace. They are +mutually exclusive, and resolution fails if you set both or neither: + +```text +invalid S3 connection: inline and reference are mutually exclusive +invalid S3 connection: neither inline nor reference is set +``` + +Use `reference` as soon as a second product talks to the same object store, so the endpoint is +defined once. + +## S3Connection + +```yaml +apiVersion: s3.kubedoop.dev/v1alpha1 +kind: S3Connection +metadata: + name: minio +spec: + host: minio.default.svc.cluster.local + port: 9000 + pathStyle: true + region: us-east-1 + credentials: + secretClass: minio-credentials +``` + +| Field | Required | Default | Notes | +|-------|----------|---------|-------| +| `host` | yes | — | Hostname of the object store. No scheme; the scheme follows `tls` | +| `port` | no | — | Omitted entirely when unset, so the endpoint uses the scheme's default port | +| `credentials` | yes | — | SecretClass supplying the access keys, see below | +| `pathStyle` | no | `false` | Addressing style, see below | +| `tls` | no | — | Presence switches the endpoint to `https` | +| `region` | no | `us-east-1` | Signing region | + +The endpoint is assembled from these fields: the scheme is `https` when `tls` is set and `http` +otherwise, so there is no separate "use TLS" switch to keep in sync. + +## Credentials + +`credentials.secretClass` names a SecretClass that must publish exactly two keys: + +| Key | Read by products as | +|-----|---------------------| +| `ACCESS_KEY` | `AWS_ACCESS_KEY_ID` | +| `SECRET_KEY` | `AWS_SECRET_ACCESS_KEY` | + +The Secret Operator mounts them as files under `/kubedoop/secret//`, and products source +them into the environment: + +```bash +export AWS_ACCESS_KEY_ID="$(cat /kubedoop/secret/s3-credentials/ACCESS_KEY)" +export AWS_SECRET_ACCESS_KEY="$(cat /kubedoop/secret/s3-credentials/SECRET_KEY)" +``` + +Credentials may additionally carry a `scope`, which narrows what the issued credential covers — +see [Authentication](../security/authentication.md) for how SecretClass scopes work. + +## pathStyle: the field that breaks MinIO + +`pathStyle` selects how the bucket is addressed: + +| Value | Resulting URL | Correct for | +|-------|---------------|-------------| +| `false` (default) | `https://.` — virtual host | AWS S3 | +| `true` | `https:///` — path | MinIO, Ceph RGW, most self-hosted backends | + +The default is right for AWS and wrong for most in-cluster deployments. MinIO in particular serves +path-style only. With the default, the client resolves `.` — `warehouse.minio` for an +in-cluster MinIO — which does not exist in DNS. + +Nothing rejects this at admission. The resource applies cleanly, the pods start, and the failure +only surfaces on the first object access. **Set `pathStyle: true` for MinIO, Ceph RGW and similar +backends.** + +## TLS + +Setting `tls` switches the endpoint to `https`. The nested `verification` decides how the server +certificate is checked: + +```yaml +spec: + host: s3.example.com + tls: + verification: + server: + caCert: + secretClass: tls # a SecretClass that issues the CA certificate +``` + +| `verification` | Behaviour | +|----------------|-----------| +| `server.caCert.secretClass` | Verify against the CA published by that SecretClass | +| `server.caCert.webPki: {}` | Verify against the system's public CA bundle | +| `none: {}` | Do not verify the certificate | + +`none` disables verification entirely and should stay out of production. + +## S3Bucket + +An `S3Bucket` is a bucket name plus a connection, and the connection is itself an +inline-or-reference pair: + +```yaml +apiVersion: s3.kubedoop.dev/v1alpha1 +kind: S3Bucket +metadata: + name: warehouse +spec: + bucketName: warehouse + connection: + reference: minio +``` + +`bucketName` is the name on the object store; the resource's own `metadata.name` is only what other +Kubedoop resources refer to. They do not have to match. + +A bucket with no `connection` fails resolution: + +```text +invalid S3 bucket "warehouse": no connection is set +``` + +## Using it from a product + +Products expose the same inline-or-reference pair. Hive Metastore takes a connection under +`clusterConfig.s3`: + +```yaml +apiVersion: hive.kubedoop.dev/v1alpha1 +kind: HiveMetastore +metadata: + name: hive +spec: + clusterConfig: + s3: + reference: minio +``` + +Swapping `reference: minio` for an `inline:` block with the same fields as an `S3Connection` spec +is equivalent, and appropriate when only one product uses that endpoint. + +## Related + +- [Roles and role groups](../common-configuration-mechanisms/roles-and-role-groups.md) +- [Resource management](./resource-manage.md) diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/core-concepts/operations/pod-disruptions.md b/i18n/zh/docusaurus-plugin-content-docs/current/core-concepts/operations/pod-disruptions.md index e69de29b..04429350 100644 --- a/i18n/zh/docusaurus-plugin-content-docs/current/core-concepts/operations/pod-disruptions.md +++ b/i18n/zh/docusaurus-plugin-content-docs/current/core-concepts/operations/pod-disruptions.md @@ -0,0 +1,82 @@ +--- +title: Pod 干扰 +--- + +Kubernetes 集群会因为和你的负载毫无关系的原因挪动 Pod：为升级排空节点、重新均衡、缩容节点池。 +这些属于**自愿性干扰**，而 +[PodDisruptionBudget](https://kubernetes.io/docs/concepts/workloads/pods/disruptions/) +正是负载用来告诉集群"我最多能同时下线几个 Pod"的手段。 + +对基于法定人数（quorum）的角色来说这尤其要紧。同时驱逐三个 ZooKeeper 节点中的两个，不是让集群变慢， +而是直接失去法定人数、停止服务。 + +Kubedoop 会**按角色**写出 PodDisruptionBudget，默认值由 Operator 根据它对该产品的了解选取。 +你不需要做任何配置就已经受到保护。 + +## 配置 + +每个角色都接受 `roleConfig.podDisruptionBudget` 配置块： + +```yaml +apiVersion: zookeeper.kubedoop.dev/v1alpha1 +kind: ZookeeperCluster +metadata: + name: zookeeper +spec: + server: + roleConfig: + podDisruptionBudget: + enabled: true + maxUnavailable: 1 + roleGroups: + default: + replicas: 3 +``` + +| 字段 | 类型 | 默认值 | 含义 | +|------|------|--------|------| +| `enabled` | 布尔 | `true` | Operator 是否为该角色写出 PDB | +| `maxUnavailable` | 整数 | 因产品而异 | 该角色允许同时下线的 Pod 数 | + +两个字段都是可选的，配置块本身也是可选的。 + +注意 `roleConfig` 是**角色级**配置，不会被下面的角色组继承或覆盖。 + +### `enabled` 默认为 true，省略时也是 + +完全不写 `podDisruptionBudget`，以及只写了 `maxUnavailable` 的配置块，两种情况下 PDB 都保持启用。 +一个角色仅仅为了调高 `maxUnavailable` 而提到这个块，它的 PDB 依然存在——只有显式写 +`enabled: false` 才会失去它。 + +### `maxUnavailable` 按产品取默认值 + +不设置时，Operator 会依据它对该产品的了解给出取值，而不是套用一个统一数字，因为安全阈值因角色而异： +三节点法定人数只能容忍损失一个，而无状态工作节点池能容忍的多得多。 + +只有当你比 Operator 更清楚该产品在你这套拓扑下的容忍度时，才需要自己设置。 + +## 使用自定义 PDB + +`enabled: false` 会让 Operator 不再为该角色写 PDB，适合你打算自己管理的场景： + +```yaml +spec: + server: + roleConfig: + podDisruptionBudget: + enabled: false +``` + +此后 Operator 不再干预该角色，你用自己的选择器创建的 PodDisruptionBudget 会正常生效。 +但要注意：关掉之后**不**补上替代品，等于该角色对自愿性驱逐完全失去保护。 + +## PDB 管不了的事 + +PodDisruptionBudget 只约束自愿性干扰。节点硬崩溃、内核 panic、Pod 因超出内存限制被杀， +这些都属于非自愿干扰，预算对它们没有发言权。应对这类情况的手段是把角色分散到不同故障域， +参见 [Pod 放置](./pod-placement.md)。 + +## 相关内容 + +- [Pod 放置](./pod-placement.md) +- [角色和角色组](../common-configuration-mechanisms/roles-and-role-groups.md) diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/core-concepts/operations/pod-placement.md b/i18n/zh/docusaurus-plugin-content-docs/current/core-concepts/operations/pod-placement.md index e69de29b..0754292c 100644 --- a/i18n/zh/docusaurus-plugin-content-docs/current/core-concepts/operations/pod-placement.md +++ b/i18n/zh/docusaurus-plugin-content-docs/current/core-concepts/operations/pod-placement.md @@ -0,0 +1,136 @@ +--- +title: Pod 放置 +--- + +角色的 Pod 落在哪里，决定了一次故障的代价有多大。三个 ZooKeeper 节点都在同一台机器上，就是三个会 +一起消失的 Pod。因此 Kubedoop 的产品都自带一份默认 +[亲和性](https://kubernetes.io/docs/concepts/scheduling-eviction/assign-pod-node/)配置， +把角色打散到不同节点；当你的拓扑需要别的安排时，也可以替换它。 + +## 默认配置已经存在 + +每个产品都为自己的角色声明了亲和性默认值——通常是一条让同一角色的副本避开同一节点的 +`podAntiAffinity`。你什么都不写就已经生效。 + +在覆盖它之前请先读下面一节，因为**覆盖不等于追加**。 + +## 配置亲和性 + +亲和性配置在 `config` 下，而 `config` 存在于两个层级： + +```yaml +apiVersion: zookeeper.kubedoop.dev/v1alpha1 +kind: ZookeeperCluster +metadata: + name: zookeeper +spec: + server: + config: + affinity: # 角色级：对该角色的所有角色组生效 + nodeAffinity: + requiredDuringSchedulingIgnoredDuringExecution: + nodeSelectorTerms: + - matchExpressions: + - key: node-role.kubernetes.io/worker + operator: Exists + roleGroups: + default: + replicas: 3 + gpu: + replicas: 2 + config: + affinity: # 角色组级：只对该组生效 + nodeAffinity: + requiredDuringSchedulingIgnoredDuringExecution: + nodeSelectorTerms: + - matchExpressions: + - key: accelerator + operator: In + values: ["nvidia"] +``` + +取值是标准的 Kubernetes +[`Affinity`](https://kubernetes.io/docs/reference/kubernetes-api/workload-resources/pod-v1/#scheduling) +对象，会原样传递到 Pod spec，因此 `nodeAffinity`、`podAffinity`、`podAntiAffinity` +的行为与在普通 Deployment 中完全一致。 + +## 亲和性是替换，不是合并 + +一共三层，自下而上： + +1. 产品的角色默认值 +2. 角色的 `config.affinity` +3. 角色组的 `config.affinity` + +**任何一层只要设置了 `affinity`，就会整体替换它下面那一层。** 这遵循 Kubernetes 的惯例，也符合 +编辑该字段的人的预期——但它带来一个很容易忽略的后果。 + +在角色组级只写一个 `nodeAffinity` 把该组钉到某种机型上，会连同产品的 `podAntiAffinity` 一起丢弃。 +结果是这组 Pod 确实落到了正确的节点上，却不再彼此打散——通常与本意正好相反。 + +**发生这种情况时不会有任何失败。** 资源是合法的，Pod spec 是合法的，所有状态条件都是绿的， +而法定人数已经悄悄不再分散了。 + +### 如何发现 + +Operator 会在集群资源上发出 Warning 事件： + +```text +Type Reason Message +Warning AffinityOverridden role "server" group "gpu": the role group's config replaces + config.affinity wholesale, discarding the podAntiAffinity declared + beneath it. config.affinity follows the Kubernetes rule and is not + merged per member; restate the discarded member alongside your own + to keep it +``` + +改动亲和性之后建议检查一下： + +```bash +kubectl get events --field-selector reason=AffinityOverridden +``` + +### 保留被替换掉的部分 + +把被丢弃的成员和你自己的配置一起重新写出来： + +```yaml +config: + affinity: + nodeAffinity: + requiredDuringSchedulingIgnoredDuringExecution: + nodeSelectorTerms: + - matchExpressions: + - key: accelerator + operator: In + values: ["nvidia"] + podAntiAffinity: # 必须重新声明，否则丢失 + preferredDuringSchedulingIgnoredDuringExecution: + - weight: 70 + podAffinityTerm: + topologyKey: kubernetes.io/hostname + labelSelector: + matchLabels: + app.kubernetes.io/name: zookeeper + app.kubernetes.io/instance: zookeeper + app.kubernetes.io/component: server +``` + +## 选择拓扑键 + +`topologyKey` 决定了"打散"是按什么维度： + +| 键 | 打散范围 | +|----|----------| +| `kubernetes.io/hostname` | 节点——防止单节点故障 | +| `topology.kubernetes.io/zone` | 可用区——防止单可用区故障 | + +按可用区打散要求每个可用区有足够的节点。如果用 +`requiredDuringSchedulingIgnoredDuringExecution` 而可用区数少于副本数，多出来的 Pod 会永远处于 +`Pending`；`preferredDuringSchedulingIgnoredDuringExecution` 则是降级而非阻塞， +这也是产品默认值偏向后者的原因。 + +## 相关内容 + +- [Pod 干扰](./pod-disruptions.md) +- [角色和角色组](../common-configuration-mechanisms/roles-and-role-groups.md) diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/core-concepts/resources/s3.md b/i18n/zh/docusaurus-plugin-content-docs/current/core-concepts/resources/s3.md index 07df9e2b..f33e70cb 100644 --- a/i18n/zh/docusaurus-plugin-content-docs/current/core-concepts/resources/s3.md +++ b/i18n/zh/docusaurus-plugin-content-docs/current/core-concepts/resources/s3.md @@ -1,2 +1,161 @@ +--- +title: S3 +--- -# S3 +Kubedoop 中很多产品都要读写对象存储：Hive 把表数据放在上面，Trino 查询它，Spark 往里写检查点。 +为了不在每个产品资源里重复填写端点和凭据，Kubedoop 把 S3 抽象成两个集群资源，由产品去引用。 + +## 两个对象 + +| 对象 | 描述内容 | +|------|----------| +| `S3Connection` | 对象存储在哪、怎么连：host、port、TLS、region、寻址风格、凭据 | +| `S3Bucket` | 一个桶名，加上这个桶所在的连接 | + +两者都属于 `s3.kubedoop.dev/v1alpha1` API 组。 + +产品用哪一个，取决于它如何寻址存储。被交付整个桶的产品用 `S3Bucket`；自己决定桶名的产品—— +比如 Hive Metastore——直接用 `S3Connection`。 + +## inline 与 reference 二选一 + +凡是接受 S3 配置的地方，Kubedoop 都提供同一对字段：`inline` 就地定义，`reference` 引用同命名空间 +下已有对象的名字。两者互斥，同时设置或都不设置都会解析失败： + +```text +invalid S3 connection: inline and reference are mutually exclusive +invalid S3 connection: neither inline nor reference is set +``` + +只要有第二个产品要访问同一个对象存储，就该改用 `reference`，让端点只定义一次。 + +## S3Connection + +```yaml +apiVersion: s3.kubedoop.dev/v1alpha1 +kind: S3Connection +metadata: + name: minio +spec: + host: minio.default.svc.cluster.local + port: 9000 + pathStyle: true + region: us-east-1 + credentials: + secretClass: minio-credentials +``` + +| 字段 | 必填 | 默认值 | 说明 | +|------|------|--------|------| +| `host` | 是 | — | 对象存储主机名。不带协议头，协议由 `tls` 决定 | +| `port` | 否 | — | 不设置时端点里完全不带端口，走协议默认端口 | +| `credentials` | 是 | — | 提供访问密钥的 SecretClass，见下文 | +| `pathStyle` | 否 | `false` | 寻址风格，见下文 | +| `tls` | 否 | — | 一旦设置，端点切换为 `https` | +| `region` | 否 | `us-east-1` | 请求签名用的区域 | + +端点由这些字段拼装：设置了 `tls` 就用 `https`，否则用 `http`。因此不存在一个需要额外同步的 +"是否启用 TLS" 开关。 + +## 凭据 + +`credentials.secretClass` 指向的 SecretClass 必须提供且仅需提供两个键： + +| 键 | 产品读取为 | +|----|------------| +| `ACCESS_KEY` | `AWS_ACCESS_KEY_ID` | +| `SECRET_KEY` | `AWS_SECRET_ACCESS_KEY` | + +Secret Operator 会把它们以文件形式挂载到 `/kubedoop/secret//`，产品再注入到环境变量： + +```bash +export AWS_ACCESS_KEY_ID="$(cat /kubedoop/secret/s3-credentials/ACCESS_KEY)" +export AWS_SECRET_ACCESS_KEY="$(cat /kubedoop/secret/s3-credentials/SECRET_KEY)" +``` + +`credentials` 还可以带 `scope`，用于收窄签发凭据的适用范围——SecretClass 的作用域机制参见 +[认证](../security/authentication.md)。 + +## pathStyle：这个字段会让 MinIO 挂掉 + +`pathStyle` 决定桶的寻址方式： + +| 取值 | 生成的 URL | 适用于 | +|------|------------|--------| +| `false`（默认） | `https://.`——虚拟主机风格 | AWS S3 | +| `true` | `https:///`——路径风格 | MinIO、Ceph RGW 等绝大多数自建后端 | + +默认值对 AWS 是对的，对大多数集群内部署是错的。MinIO 尤其只支持路径风格。用默认值时客户端会去解析 +`.`——集群内的 MinIO 就是 `warehouse.minio`——这个域名在 DNS 里并不存在。 + +**准入阶段不会拦截这个错误。** 资源正常创建，Pod 正常启动，直到第一次访问对象时才失败。 +**MinIO、Ceph RGW 这类后端请设置 `pathStyle: true`。** + +## TLS + +设置 `tls` 会把端点切到 `https`。嵌套的 `verification` 决定如何校验服务端证书： + +```yaml +spec: + host: s3.example.com + tls: + verification: + server: + caCert: + secretClass: tls # 签发 CA 证书的 SecretClass +``` + +| `verification` | 行为 | +|----------------|------| +| `server.caCert.secretClass` | 用该 SecretClass 提供的 CA 校验 | +| `server.caCert.webPki: {}` | 用系统内置的公共 CA 集校验 | +| `none: {}` | 完全不校验证书 | + +`none` 会彻底关闭校验，不应出现在生产环境。 + +## S3Bucket + +`S3Bucket` 就是一个桶名加一个连接，而这个连接本身同样是 inline/reference 二选一： + +```yaml +apiVersion: s3.kubedoop.dev/v1alpha1 +kind: S3Bucket +metadata: + name: warehouse +spec: + bucketName: warehouse + connection: + reference: minio +``` + +`bucketName` 是对象存储上的真实桶名；资源自身的 `metadata.name` 只是其他 Kubedoop 资源引用它时 +用的名字。两者不必相同。 + +没有 `connection` 的桶会解析失败： + +```text +invalid S3 bucket "warehouse": no connection is set +``` + +## 在产品中使用 + +产品侧暴露的是同一对 inline/reference 字段。Hive Metastore 在 `clusterConfig.s3` 下接受一个连接： + +```yaml +apiVersion: hive.kubedoop.dev/v1alpha1 +kind: HiveMetastore +metadata: + name: hive +spec: + clusterConfig: + s3: + reference: minio +``` + +把 `reference: minio` 换成字段与 `S3Connection` spec 相同的 `inline:` 块，效果等价，适合只有一个 +产品使用该端点的场景。 + +## 相关内容 + +- [角色和角色组](../common-configuration-mechanisms/roles-and-role-groups.md) +- [资源管理](./resource-manage.md)