diff --git a/elasticsearch.md b/elasticsearch.md index efabfda..039938d 100644 --- a/elasticsearch.md +++ b/elasticsearch.md @@ -473,14 +473,29 @@ readonlyrest.ssl.key_alias: "my-server-cert" # optional; if omitted, RO readonlyrest.ssl.allowed_protocols: [TLSv1.2, TLSv1.3] readonlyrest.ssl.allowed_ciphers: [TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256, TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256] -# Optional: mutual TLS — require clients to present a certificate -readonlyrest.ssl.client_authentication: false # default +# Optional: mutual TLS — ask clients to present a certificate +readonlyrest.ssl.client_authentication: none # none (default) | optional | required # Optional: custom trust anchor for client certificates (defaults to JVM truststore) readonlyrest.ssl.truststore_file: "truststore.jks" readonlyrest.ssl.truststore_pass: "" ``` +`client_authentication` accepts: + +| Value | Meaning | +| --- | --- | +| `none` | No certificate is requested. The default. | +| `optional` | A certificate is requested and verified if presented; clients without one still connect. | +| `required` | A certificate is mandatory; the TLS handshake fails without a valid one. | + +Use `optional` when certificate-bearing services and password-bearing users share the same port. Use +`required` when every client holds a certificate — it moves the rejection from the ACL to the handshake, +so a client without one sees a TLS error rather than an HTTP response. + +Once clients present certificates, the [`pkis` connector](elasticsearch.md#pki-connector) can turn them +into ReadonlyREST users, so a service authenticates with nothing but its certificate. + **PEM option (preferred):** ```yaml http.type: ssl_netty4 @@ -573,12 +588,13 @@ readonlyrest.ssl_internode.hostname_verification: true By default, the server does not request a client certificate. When enabled, Elasticsearch verifies the client's identity via mutual TLS. -For external REST API: +For the external REST API, the setting takes `none`, `optional` or `required` — see the value table under +[External REST API](elasticsearch.md#external-rest-api): ```yaml -readonlyrest.ssl.client_authentication: true +readonlyrest.ssl.client_authentication: required ``` -For internode communication: +For internode communication it is a boolean: ```yaml readonlyrest.ssl_internode.client_authentication: true ``` @@ -1207,6 +1223,91 @@ See the dedicated [LDAP section](elasticsearch.md#ldap-connector) * Groups logic syntax can be uses as part of this rule, as described in the [Checking groups logic section](details/authorization-rules-details.md#checking-groups-logic) * For more information on the ROR's authorization rules, see [Authorization rules details](details/authorization-rules-details.md) +##### `pki_authentication` + +Authenticates a client by the TLS client certificate its connection was established with, using the +configured PKI provider (here `corporate_pki`). See the [PKI connector section](elasticsearch.md#pki-connector) +to learn how to configure the provider. + +simple version: +`pki_authentication: corporate_pki` + +extended version: +```yaml +pki_authentication: + name: corporate_pki + users: ["svc-logstash", "beats-*"] # optional: only accept these usernames +``` + +- The certificate is the only credential. No password, token or header is involved. +- The certificate has already been verified by whichever component terminated TLS, so this rule never + decides whether a certificate is trustworthy — only who it identifies. +- A request that arrives **without** a certificate does not match, and evaluation moves on to the next + block. That is what lets certificate-bearing services and password-bearing users share one port — + but only where client authentication is `optional`. With `required`, a client holding no certificate + is rejected during the TLS handshake and never reaches the ACL at all. + +##### `pki_authorization` + +```yaml +pki_authorization: + name: "corporate_pki" + groups_any_of: ["ingest_services"] +``` + +- It reads the groups out of the client certificate, using the `groups` section of the configured PKI + provider. The provider must define one, or the settings are rejected at startup. +- The groups are *external* groups, so they flow into the usual + [external to local groups mapping](details/groups-rule-mapping.md). Values that no mapping matches + are simply discarded, which is how you keep certificate attributes that are not roles (`OU=Warsaw`, + `OU=IT`) out of ReadonlyREST. +- Groups logic syntax can be used as part of this rule, as described in the + [Checking groups logic section](details/authorization-rules-details.md#checking-groups-logic) + +##### `pki_auth` + +Shorthand rule that combines `pki_authentication` and `pki_authorization` together. + +```yaml +pki_auth: + name: "corporate_pki" + groups_any_of: ["ingest_services"] +``` + +The same can be written as: + +```yaml +pki_authentication: corporate_pki +pki_authorization: + name: "corporate_pki" + groups_any_of: ["ingest_services"] +``` + +The most common enterprise arrangement, however, takes the identity from the certificate and the groups +from the corporate directory: + +```yaml +- name: "Platform services" + pki_authentication: + name: "corporate_pki" + ldap_authorization: + name: "corporate_ldap" + groups_any_of: ["platform_services"] + indices: ["logs-*"] +``` + +**⚠️IMPORTANT** for this to work, the username the certificate yields has to be the one the directory +can look up. If your CA puts a display name in the `CN` but LDAP keys on `uid` or `sAMAccountName`, the +LDAP search fails with a confusing "user not found". See +[reading the identity](elasticsearch.md#reading-the-identity) for how to extract the right value without +re-issuing certificates. + +* [Impersonation](details/impersonation.md) works as it does for `proxy_auth`: when the rule declares a + `users` list the impersonated user is checked against it, and without one impersonation is refused for + that rule. Certificate-derived groups can never be impersonated, because an impersonator presents no + certificate belonging to the impersonated user. +* For more information on the ROR's authorization rules, see [Authorization rules details](details/authorization-rules-details.md) + ##### `jwt_authentication` See below, the dedicated [JSON Web Tokens section](elasticsearch.md#json-web-token-jwt-auth). It's an authentication rule. @@ -2723,6 +2824,192 @@ readonlyrest: search_groups_base_DN: "ou=Groups,dc=example2,dc=com" ``` +#### PKI connector + +A PKI provider turns a TLS client certificate into a ReadonlyREST user. It is how a service +authenticates with nothing but the certificate it already holds — no password in a `logstash.yml`, no +token on disk. + +Providers are declared in the `pkis` section and referenced by name from the +[`pki_authentication`](elasticsearch.md#pki_authentication), +[`pki_authorization`](elasticsearch.md#pki_authorization) and +[`pki_auth`](elasticsearch.md#pki_auth) rules. + +```yaml +readonlyrest: + + access_control_rules: + + - name: "Logstash ingest" + pki_authentication: + name: "corporate_pki" + users: ["logstash-*"] + actions: ["indices:data/write/*"] + indices: ["ingest-*"] + + pkis: + - name: corporate_pki + users: + user_id_attribute: "CN" # the default; shown for clarity +``` + +##### Before you start + +ReadonlyREST can only read a certificate that was presented to **this** Elasticsearch node, so: + +- **TLS has to terminate at Elasticsearch.** If a load balancer, ingress or service mesh terminates it + upstream, no certificate reaches ReadonlyREST and PKI rules never match. This is the single most common + reason PKI "does not work". Such deployments have to move TLS termination, or keep using `proxy_auth`. +- **Client authentication has to be on.** Either `readonlyrest.ssl.client_authentication` (see + [Encryption](elasticsearch.md#encryption)) or `xpack.security.http.ssl.client_authentication`, set to + `optional` or `required`. With `none` the node never asks for a certificate, so PKI rules never match. + ReadonlyREST cannot detect this for you: the ACL has no view of your TLS configuration. +- **Elasticsearch 7.0 or newer.** On 6.7 a request cannot be traced back to the connection it arrived on. + +**⚠️IMPORTANT** PKI is only as strong as the verification your TLS layer performs. A setting such as +`xpack.security.http.ssl.verification_mode: none` makes Elasticsearch request a certificate and then +validate nothing, so a certificate from *any* CA is accepted — and anyone able to run a CA can issue one +carrying `CN=svc-logstash` and be authenticated as that service. ReadonlyREST cannot tell the difference: +by the time a rule sees the certificate it looks legitimate. Always verify chains. Pinning `issuer_dn` +(below) is defence in depth, not a replacement: the issuer is a field *inside* the presented certificate, +so with chain validation disabled a forged certificate simply carries a matching one. It constrains +certificates your TLS layer has already verified — it cannot make an unverified one trustworthy. + +##### Reading the identity + +The `users` section says how to get a username out of a certificate. It follows the same shape as the +LDAP connector: an optional `mode` selecting a variant, and a default that needs no `mode` at all. + +| `mode` | Keys | What it does | +| --- | --- | --- | +| `subject_dn_attribute` *(default)* | `user_id_attribute` | Reads a named attribute out of the subject DN. Defaults to `CN`. | +| `subject_dn_pattern` | `pattern` | Applies a regular expression to the subject DN. | +| `san` | `san_type`, optional `pattern` | Reads a Subject Alternative Name entry. | + +Attribute names are matched case-insensitively, and the DN is parsed structurally rather than as a +string, so escaping and multi-valued RDNs are handled for you: `CN=Smith\, John` yields `Smith, John`. + +**Active Directory.** In AD the login identity is the userPrincipalName, not the `CN` — a certificate +with `CN=John Smith` and UPN `jsmith@corp.example.com` gives an unusable username under `CN`, and the +login name is not in the DN at all. Read the UPN instead: + +```yaml + pkis: + - name: ad_pki + users: + mode: san + san_type: upn # the otherName entry with OID 1.3.6.1.4.1.311.20.2.3 +``` + +`san_type` accepts `dns`, `email`, `uri`, `ip` and `upn`. Where a certificate carries several entries of +the requested type — routine for host certificates, which often hold an FQDN, a short name and aliases — +the first is used, and an optional `pattern` selects among them: + +```yaml + pkis: + - name: host_pki + users: + mode: san + san_type: dns + pattern: "^(.+)\\.corp\\.example\\.com$" +``` + +For a DN your CA lays out unusually, the regular expression escape hatch applies to the RFC 2253 +rendering of the subject, and captured values are unescaped for you: + +```yaml + pkis: + - name: legacy_pki + users: + mode: subject_dn_pattern + pattern: "^CN=([^,]+),OU=Service Accounts,.*$" +``` + +A pattern has to compile, define exactly one capture group, and be unable to match an empty value — +all checked when the settings load, not when a request arrives. + +##### Reading groups + +An optional `groups` section makes the certificate carry authorization too, which suits clusters with no +directory to consult: + +```yaml + pkis: + - name: internal_pki + users: + user_id_attribute: "CN" + groups: + group_id_attribute: "OU" +``` + +The same three modes are available. Where a username takes the first value found, groups take all of +them — so one `subject_dn_pattern` serves both, and `pattern: "OU=grp-([^,]+)"` yields a group per match. + +Note that a DN's OUs are frequently a position in an org tree rather than a set of roles: + +```text +CN=beats-01,OU=ingest,O=Corp -> ["ingest"] +CN=jsmith,OU=Engineering,OU=EMEA,OU=Employees -> ["Engineering", "EMEA", "Employees"] +``` + +ReadonlyREST does not interpret their order or hierarchy, and offers no filter — mapping only the +role-bearing ones is your decision, expressed in the +[groups mapping](details/groups-rule-mapping.md). Anything you do not map is discarded. + +Group *names* are not supported: a certificate carries a single value per attribute, so there is no +second attribute to read a display name from. `group_name_attribute` is rejected at startup rather than +silently ignored. + +##### Restricting which certificates a provider speaks for + +One corporate CA usually issues to more than one population: + +```text +CN=svc-logstash,OU=Services,DC=corp,DC=example,DC=com <- machine +CN=John Smith,OU=People,DC=corp,DC=example,DC=com <- human +``` + +Both chain to the same CA, so TLS trusts them equally and a plain `CN` extractor would authenticate +both. Two optional constraints keep them apart. They are checked before anything is extracted, so a +certificate out of scope yields neither a username nor groups. + +```yaml + pkis: + - name: services_pki + subject_dn_base: "OU=Services,DC=corp,DC=example,DC=com" # subject must end with this + issuer_dn: "CN=Corp Issuing CA,DC=corp,DC=example,DC=com" # issuer must match exactly + users: + user_id_attribute: "CN" + + - name: employees_pki + subject_dn_base: "OU=People,DC=corp,DC=example,DC=com" + users: + user_id_attribute: "CN" +``` + +`issuer_dn` covers what `subject_dn_base` cannot: any CA your node trusts can mint any name, so where a +truststore holds more than one CA, pinning the issuer is what stops one CA impersonating identities from +another. + +##### Migrating from the X-Pack PKI realm + +The concepts line up. X-Pack derives a username from the subject DN with `username_pattern` and resolves +roles through role mappings; ReadonlyREST derives one with `mode`/`user_id_attribute`/`pattern` and +resolves groups from the `users` section, LDAP, or the certificate itself. An X-Pack `username_pattern` +regex can be carried over as-is under `mode: subject_dn_pattern`. + +##### Notes + +- Certificate revocation (CRL/OCSP) is not consulted — Elasticsearch does not support it either. To + deprovision a compromised identity, remove it from the rule's `users` allowlist, or from the top-level + `users` definitions section so that it resolves no groups, and reload the settings. The provider's own + `users` section only says how to read a username out of a certificate — it grants nothing, so editing + it deprovisions nobody. +- The certificate belongs to the connection, so every request multiplexed over one keep-alive connection + carries the same identity. The ACL is still evaluated per request against the settings in force at that + moment, so a reload takes effect on the next request, existing connections included — there is no need + to close them. + #### External Basic Auth ReadonlyREST will forward the received `Authorization` header to a website of choice and evaluate the returned HTTP status code to verify the provided credentials. This is useful if you already have a web server with all the credentials configured and the credentials are passed over the `Authorization` header.