Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 9 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -140,6 +140,15 @@ but container releases use the upstream-derived format documented in
- Fixed a race in the reload test that read `/proc/PID/status` for a worker
that had already exited, which aborted the listing instead of skipping the
vanished process.
- Added a qualified preview ClickHouse HTTP proxying profile that bounds
methods, body size, timeouts, and temporary storage, reports failures by
exception code rather than by statement, and records `NONE` upstream fields
for a request the proxy refused; tests prove a password and column name sent
as request parameters reach neither the access stream nor the error stream.
- Stated that the ClickHouse proxy is not an authorization boundary, because
NGINX parses HTTP rather than SQL and cannot enforce read-only access,
restrict statements, or bound returned rows; those remain ClickHouse
`readonly` settings, quotas, row policies, and grants.
- Added a qualified preview health-endpoint profile separating
upstream-independent liveness from upstream-reflecting readiness, logging
only failing probes, and exposing `stub_status` on a separate listener that
Expand Down
83 changes: 79 additions & 4 deletions docs/CONFIGURATION-PROFILES.md
Original file line number Diff line number Diff line change
@@ -1,10 +1,10 @@
# Qualified HTTP and TLS configuration profiles

The repository provides minimal static-serving, HTTP reverse-proxy, HTTP
load-balancing, WebSocket-proxying, request-limiting, and health-endpoint
configurations under `examples/profiles`. Static serving, HTTP reverse proxy,
HTTP load balancing, WebSocket proxying, request and connection limiting,
health and readiness endpoints,
load-balancing, WebSocket-proxying, request-limiting, health-endpoint, and
ClickHouse-proxying configurations under `examples/profiles`. Static serving,
HTTP reverse proxy, HTTP load balancing, WebSocket proxying, request and
connection limiting, health and readiness endpoints, ClickHouse HTTP proxying,
TLS termination, mutual TLS, and verified HTTPS upstream profiles are exercised
on native AMD64 and ARM64 runners with rootless Podman and then with Docker
compatibility execution. They remain **preview/unqualified** until an immutable
Expand Down Expand Up @@ -62,6 +62,81 @@ Its access-event schema is:
| `body_bytes_sent` | integer | Response-body bytes sent. |
| `request_time` | number | Total request duration in seconds. |

## ClickHouse HTTP proxying

[`examples/profiles/clickhouse/nginx.conf`](../examples/profiles/clickhouse/nginx.conf)
proxies the ClickHouse HTTP interface.

### This proxy is not an authorization boundary

NGINX parses HTTP, not SQL. It cannot make a session read-only, restrict which
statements run, bound how many rows a query returns, or tell a `SELECT` from a
`DROP`.

Anything that depends on the content of a query belongs to ClickHouse:
`readonly` user settings, quotas, row policies, and per-user grants. Treating
this file as the control that stops clients modifying data is a misreading of
what it does. The profile restricts methods and body size, which bounds the
shape of a request, not its meaning.

### Credentials and query text in the request target

This is the profile where the repository-wide logging rule earns its keep.

The ClickHouse HTTP interface accepts `?query=...`, and many deployments accept
`?user=...&password=...`. The request target therefore routinely carries both
credentials and customer data. Every profile here logs `$uri` and never
`$request`, `$request_uri`, or `$args`, so passwords and query text are
structurally absent from the access stream rather than filtered out of it.

The test asserts this directly: it issues a request carrying a password and a
column name as parameters, then requires that neither string appears anywhere
in the container's output.

An operator should confirm the same property for anything downstream. A log
collector, an error tracker, or a reverse proxy in front that records full
request lines reintroduces exactly what this profile removes.

### Failures are reported by code, not by statement

`X-ClickHouse-Exception-Code` is recorded as a structured field, which gives
the failure class without putting any part of the query into the log. A request
the proxy refuses never reaches the database, so its upstream fields record
`NONE`, keeping "the database rejected this" distinguishable from "the database
never saw it".

### Temporary storage

`proxy_max_temp_file_size 0` with buffering off is deliberate. A result set
larger than the proxy buffers would otherwise spill to `proxy_temp_path`, which
is on the container's `/tmp` tmpfs. One large result could consume the tmpfs
and take the container down, so the response is streamed instead.

### Timeouts and capacity

Analytical queries run far longer than web requests, so the read timeout is
large and scoped to the query location. Each waiting query holds a worker
connection, so size `worker_connections` for the expected concurrent query
count rather than for request rate. Retries are disabled: re-running a
statement is unsafe for anything that writes and wasteful for anything else.

### Qualified behaviour

The tests run against a stand-in for the ClickHouse HTTP interface built from
this image. It reports what the proxy forwarded and can answer with an
exception code; it executes nothing and implements no part of the ClickHouse
protocol.

They prove that a POST body reaches the upstream, that credentials and query
text in the request target never reach the log, that an exception code is
recorded, that methods outside the interface are refused, and that an oversized
body is refused before reaching the database.

They do **not** qualify interoperation with a real ClickHouse server, protocol
or version behaviour, query semantics, compression negotiation, or performance
against real result sets. A deployment must qualify those against its own
server.

## Extended health and readiness endpoints

[`examples/profiles/health/nginx.conf`](../examples/profiles/health/nginx.conf)
Expand Down
6 changes: 3 additions & 3 deletions docs/LOGGING.md
Original file line number Diff line number Diff line change
Expand Up @@ -84,9 +84,9 @@ and [`container/conf.d/default.conf`](../container/conf.d/default.conf).
## Qualified preview HTTP and TLS formats

The static, HTTP reverse-proxy, HTTP load-balancing, WebSocket-proxying,
request-limiting, health-endpoint, TLS termination, mutual-TLS, and
verified-upstream examples implement the safer structured contract described
above.
request-limiting, health-endpoint, ClickHouse-proxying, TLS termination,
mutual-TLS, and verified-upstream examples implement the safer structured
contract described above.
They use JSON escaping, validate a bounded
`X-Request-ID` or generate `$request_id`, and log `$uri` rather than the query-
bearing request target. Tests parse every emitted access event, exercise JSON
Expand Down
2 changes: 0 additions & 2 deletions docs/ROADMAP.md
Original file line number Diff line number Diff line change
Expand Up @@ -70,12 +70,10 @@ parallel only where it does not assume an unfrozen NGINX package or module set.

## Package 3: supported configurations and TLS

- [ ] Provide a tested, minimal example for ClickHouse HTTP proxying.
- [ ] Extend the qualified defaults to DNS resolution, upstream verification,
and the remaining profiles without silently breaking general-purpose use.
- [ ] Document configuration mounting, validation, reload, rollback, logging,
troubleshooting, and secret redaction.
- [ ] Implement and test structured logging for ClickHouse.
- [ ] Qualify runtime collection, rotation ownership, pipeline failure, and
retention evidence for the selected logging platform.
- [ ] Qualify lifecycle-alert delivery and exact platform cryptographic-policy
Expand Down
1 change: 1 addition & 0 deletions docs/SUPPORT.md
Original file line number Diff line number Diff line change
Expand Up @@ -31,6 +31,7 @@ Absence from a matrix means unqualified, not implicitly compatible.
| WebSocket proxying profile | Preview/unqualified | Upgrade forwarding, derived connection disposition, `101` relay, and unaffected plain HTTP are tested; frame exchange, session duration, and concurrent session capacity are not. |
| Request and connection limiting profile | Preview/unqualified | Rate rejection, connection rejection, bandwidth pacing, and unlimited health checks are tested; workload tuning, zone sizing, and exhausted-zone behaviour are not. |
| Health and readiness profile | Preview/unqualified | Upstream-independent liveness, upstream-reflecting readiness, failure-only probe logging, and a default-deny status surface are tested; probe tuning, startup probes, and an allow-listed status surface are not. |
| ClickHouse HTTP proxying profile | Preview/unqualified | Method and body bounds, credential and query-text exclusion from logs, exception-code reporting, and bounded temporary storage are tested against a stand-in; interoperation with a real ClickHouse server is not qualified. |
| TLS termination and mTLS profiles | Preview/unqualified | TLS 1.2/1.3, client authentication, leaf renewal, CRL enforcement, and negative cases are tested; production PKI operations and exact-host cryptographic policy remain unqualified. |
| Verified HTTPS upstream profile | Preview/unqualified | Chain, hostname, SNI, revocation, overlapping-CA rotation, and restricted-runtime behavior are tested; deployment DNS, egress, and PKI remain operator-owned. |
| Linux AMD64 and ARM64 | Preview/unqualified | Native CI exists; release-candidate evidence is not complete. |
Expand Down
6 changes: 3 additions & 3 deletions docs/USE-CASES.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,9 +8,9 @@ against the released image.

The development image has tested preview profiles for static content, HTTP and
verified-HTTPS upstream proxying, HTTP load balancing, WebSocket proxying,
request and connection limiting, health and readiness endpoints, TLS
termination, and mutual TLS, including structured logging on unprivileged
ports. The other
request and connection limiting, health and readiness endpoints, ClickHouse
HTTP proxying, TLS termination, and mutual TLS, including structured logging on
unprivileged ports. The other
profiles below are design targets for the first release unless stated
otherwise. See [Qualified HTTP and TLS configuration profiles](CONFIGURATION-PROFILES.md)
for the exact implemented boundary.
Expand Down
176 changes: 176 additions & 0 deletions examples/profiles/clickhouse/nginx.conf
Original file line number Diff line number Diff line change
@@ -0,0 +1,176 @@
worker_processes auto;
pid /tmp/nginx.pid;
error_log /dev/stderr notice;

events {
worker_connections 1024;
}

http {
include /etc/nginx/mime.types;
default_type application/octet-stream;

map $http_x_request_id $correlation_id {
"~^[A-Za-z0-9][A-Za-z0-9._-]{0,63}$" $http_x_request_id;
default $request_id;
}

# A connection that never produced a request has no method: a failed TLS
# handshake, a malformed request line, or a client that disconnects before
# the request is read. Those are not application requests, so they stay in
# the error stream instead of emitting a structured access event with empty
# fields.
map $request_method $is_application_request {
default 1;
"" 0;
}

# ClickHouse reports failures with a numeric exception code in a response
# header. Recording it gives operators the failure class without putting
# any part of the query text into the log.
map $upstream_http_x_clickhouse_exception_code $clickhouse_exception_code {
default $upstream_http_x_clickhouse_exception_code;
"" NONE;
}

# A request the proxy refuses never reaches an upstream, so every upstream
# variable is empty. Naming that case keeps each field in the access event
# non-empty and keeps "the database rejected this" distinguishable from
# "the database never saw it", which is the difference between a query
# problem and a gateway problem.
map $upstream_addr $upstream_addr_value {
default $upstream_addr;
"" NONE;
}

map $upstream_status $upstream_status_value {
default $upstream_status;
"" NONE;
}

map $upstream_connect_time $upstream_connect_time_value {
default $upstream_connect_time;
"" NONE;
}

map $upstream_header_time $upstream_header_time_value {
default $upstream_header_time;
"" NONE;
}

map $upstream_response_time $upstream_response_time_value {
default $upstream_response_time;
"" NONE;
}

# The access event records $uri and never $request, $request_uri, or $args.
#
# That rule matters more here than anywhere else in this repository. The
# ClickHouse HTTP interface accepts `?query=...` and, on many deployments,
# `?user=...&password=...`, so the request target routinely carries both
# credentials and customer data. Logging the request line would write
# passwords and query text into the access stream, from where they spread
# into collectors, retention, and backups.
log_format profile_json escape=json
'{"timestamp":"$time_iso8601",'
'"request_id":"$correlation_id",'
'"method":"$request_method",'
'"uri":"$uri",'
'"protocol":"$server_protocol",'
'"status":$status,'
'"body_bytes_sent":$body_bytes_sent,'
'"request_time":$request_time,'
'"clickhouse_exception_code":"$clickhouse_exception_code",'
'"upstream_addr":"$upstream_addr_value",'
'"upstream_status":"$upstream_status_value",'
'"upstream_connect_time":"$upstream_connect_time_value",'
'"upstream_header_time":"$upstream_header_time_value",'
'"upstream_response_time":"$upstream_response_time_value"}';
access_log /dev/stdout profile_json if=$is_application_request;

client_body_temp_path /tmp/nginx-client-body;
proxy_temp_path /tmp/nginx-proxy;
fastcgi_temp_path /tmp/nginx-fastcgi;
uwsgi_temp_path /tmp/nginx-uwsgi;
scgi_temp_path /tmp/nginx-scgi;

server_tokens off;
client_body_timeout 10s;
client_header_timeout 10s;
keepalive_timeout 30s;
send_timeout 30s;

# Replace this endpoint with the deployment's approved service name.
upstream clickhouse_backend {
server backend:8123;
keepalive 8;
}

server {
listen 8080 default_server;
listen [::]:8080 default_server;
server_name _;

add_header X-Content-Type-Options nosniff always;
add_header X-Request-ID $correlation_id always;

# This proxy is not an authorization boundary.
#
# NGINX parses HTTP, not SQL. It cannot make a session read-only,
# restrict which statements run, bound how many rows a query returns,
# or tell a SELECT from a DROP. Anything that depends on the content of
# a query belongs to ClickHouse: `readonly` user settings, quotas, row
# policies, and per-user grants. Treating this file as the control that
# keeps clients from modifying data is a misreading of what it does.
location = /query {
proxy_pass http://clickhouse_backend/;
proxy_http_version 1.1;
proxy_set_header Connection "";
proxy_set_header Host $proxy_host;
proxy_set_header X-Forwarded-For $remote_addr;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_set_header X-Request-ID $correlation_id;

# Queries are sent as a request body. Inserts need a deliberately
# larger value; raise it with a stated reason rather than removing
# the bound, because an unbounded body lets one client occupy
# temporary storage on the way to the database.
client_max_body_size 10m;

# Analytical queries run far longer than web requests. The timeout
# is scoped to this location so an ordinary endpoint cannot inherit
# it, and each waiting query still holds a worker connection, so
# size `worker_connections` for the expected concurrent query count
# rather than for request rate.
proxy_connect_timeout 5s;
proxy_send_timeout 300s;
proxy_read_timeout 300s;

# A result set larger than the proxy buffers would otherwise spill
# to `proxy_temp_path`, which is on the container's `/tmp` tmpfs.
# A single large result could then consume the tmpfs and take the
# container down with it. Streaming the response to the client
# instead keeps memory and temporary storage bounded.
proxy_max_temp_file_size 0;
proxy_buffering off;

# A retry would re-execute the statement. That is unsafe for
# anything that writes, and wasteful for anything that does not.
proxy_next_upstream off;

limit_except GET HEAD POST {
deny all;
}
}

location = /healthz {
access_log off;
default_type text/plain;
return 200 "ok\n";
}

location / {
return 404;
}
}
}
42 changes: 42 additions & 0 deletions tests/fixtures/profile-clickhouse/nginx.conf
Original file line number Diff line number Diff line change
@@ -0,0 +1,42 @@
worker_processes 1;
pid /tmp/nginx.pid;
error_log /dev/stderr notice;

events {
worker_connections 128;
}

http {
access_log off;
client_body_temp_path /tmp/nginx-client-body;
proxy_temp_path /tmp/nginx-proxy;
fastcgi_temp_path /tmp/nginx-fastcgi;
uwsgi_temp_path /tmp/nginx-uwsgi;
scgi_temp_path /tmp/nginx-scgi;

server {
listen 8123;
server_name _;

client_max_body_size 32m;

# This fixture stands in for the ClickHouse HTTP interface. It reports
# what the proxy forwarded and can return an exception code on demand,
# which covers the part the proxy profile owns. It executes nothing and
# implements no part of the ClickHouse protocol.
#
# A request carrying `fail=1` answers like a ClickHouse error so the
# proxy's handling of the exception-code header can be exercised.
location / {
default_type application/json;

if ($arg_fail = 1) {
add_header X-ClickHouse-Exception-Code 241 always;
return 500 '{"error":"stand-in exception"}';
}

add_header X-ClickHouse-Summary '{"read_rows":"0"}' always;
return 200 '{"request_id":"$http_x_request_id","method":"$request_method","forwarded_proto":"$http_x_forwarded_proto","content_length":"$http_content_length"}\n';
}
}
}
Loading
Loading