Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 10 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -140,6 +140,16 @@ but container releases use the upstream-derived format documented in
- Fixed a race in the reload test that read `/proc/PID/status` for a worker
that had already exited, which aborted the listing instead of skipping the
vanished process.
- Added a qualified preview request-limiting profile applying independent
request-rate, concurrent-connection, and per-connection bandwidth budgets,
answering `429` rather than the default `503`, keeping the health endpoint
outside every limit, and recording limit outcomes without recording the limit
key; tests prove rate rejection, connection rejection specifically, and that
health checks keep answering while a client's request budget is exhausted.
- Documented that a limit keyed on the direct peer address becomes a global cap
behind a proxy, that keying on a client-controlled header removes the limit
entirely, and that `limit_req` is evaluated before `limit_conn` so a
rate-rejected request records the connection limit as not evaluated.
- Added a qualified preview WebSocket-proxying profile that derives the
upstream connection disposition from a map rather than copying it from the
client, scopes the long idle timeout to the upgrade location, and never
Expand Down
74 changes: 72 additions & 2 deletions docs/CONFIGURATION-PROFILES.md
Original file line number Diff line number Diff line change
@@ -1,9 +1,9 @@
# Qualified HTTP and TLS configuration profiles

The repository provides minimal static-serving, HTTP reverse-proxy, HTTP
load-balancing, and WebSocket-proxying configurations under
load-balancing, WebSocket-proxying, and request-limiting configurations under
`examples/profiles`. Static serving, HTTP reverse proxy, HTTP load balancing,
WebSocket proxying,
WebSocket proxying, request and connection limiting,
TLS termination, mutual TLS, and verified HTTPS upstream profiles are exercised
on native AMD64 and ARM64 runners with rootless Podman and then with Docker
compatibility execution. They remain **preview/unqualified** until an immutable
Expand Down Expand Up @@ -61,6 +61,76 @@ Its access-event schema is:
| `body_bytes_sent` | integer | Response-body bytes sent. |
| `request_time` | number | Total request duration in seconds. |

## Request-rate and connection limiting

[`examples/profiles/rate-limited/nginx.conf`](../examples/profiles/rate-limited/nginx.conf)
applies three separate budgets to a served tree: request rate, concurrent
connections, and per-connection bandwidth. They are independent, and a
deployment that sets only one leaves the others unbounded.

### The limit key decides whether the limit exists

The key is `$binary_remote_addr`, the direct peer address.

Behind a load balancer or ingress controller that address is the *proxy*, so
every client shares one bucket and a per-client limit silently becomes a global
cap. A deployment in that position needs a key derived from a forwarded address
it actually trusts, through `realip` with a trusted-proxy list or an equivalent
reviewed mechanism.

Never key a limit on a header the client controls. A client that chooses its
own key gets a fresh bucket for every request, and the limit stops existing
while continuing to look configured.

The zones are shared across workers and are sized in advance. An exhausted zone
fails closed and rejects new clients, so size for the expected distinct-client
count rather than for steady-state traffic.

### Status code

Both limits answer `429`. The NGINX default is `503`, which is
indistinguishable from an outage and invites clients to retry harder against a
server that is already shedding load.

### Evaluation order matters when reading logs

`limit_req` is evaluated before `limit_conn`. Once the rate limit is rejecting,
the connection limit is never reached, and its field records `NOT_EVALUATED`
rather than `PASSED`. A reader who treats `NOT_EVALUATED` as "allowed" will
conclude the connection limit is inactive when it is simply downstream of a
limit that is already firing.

### The health endpoint is outside both limits

A limited health endpoint turns a traffic spike into a failed liveness probe
and a restart, which removes capacity exactly when it is needed.

### Access-event schema

The profile emits the common fields plus:

| Field | JSON type | Meaning |
| --- | --- | --- |
| `limit_req_result` | string | `PASSED`, `DELAYED`, `REJECTED`, a dry-run variant, or `NOT_EVALUATED`. |
| `limit_conn_result` | string | `PASSED`, `REJECTED`, a dry-run variant, or `NOT_EVALUATED`. |

The limit *key* is deliberately not logged. Recording the outcome supports
capacity and abuse analysis; recording the raw client identifier for every
request adds a personal identifier to an access stream that is otherwise free
of them.

### Qualified behaviour

The tests prove that a request inside both budgets passes, that exhausting the
request rate produces `429` recorded as `REJECTED`, that concurrent requests
beyond the connection maximum are rejected by the connection limit
specifically, and that the health endpoint keeps answering while the client's
request budget is exhausted.

They do **not** qualify tuning for any particular workload, zone sizing under
real client populations, behaviour once a zone is exhausted, or the interaction
between these limits and an upstream rate limiter.

## WebSocket proxying

[`examples/profiles/websocket/nginx.conf`](../examples/profiles/websocket/nginx.conf)
Expand Down
6 changes: 3 additions & 3 deletions docs/LOGGING.md
Original file line number Diff line number Diff line change
Expand Up @@ -83,9 +83,9 @@ and [`container/conf.d/default.conf`](../container/conf.d/default.conf).

## Qualified preview HTTP and TLS formats

The static, HTTP reverse-proxy, HTTP load-balancing, WebSocket-proxying, TLS
termination, mutual-TLS, and verified-upstream examples implement the safer
structured contract described above.
The static, HTTP reverse-proxy, HTTP load-balancing, WebSocket-proxying,
request-limiting, TLS termination, mutual-TLS, and verified-upstream examples
implement the safer structured contract described above.
They use JSON escaping, validate a bounded
`X-Request-ID` or generate `$request_id`, and log `$uri` rather than the query-
bearing request target. Tests parse every emitted access event, exercise JSON
Expand Down
6 changes: 3 additions & 3 deletions docs/ROADMAP.md
Original file line number Diff line number Diff line change
Expand Up @@ -71,13 +71,13 @@ parallel only where it does not assume an unfrozen NGINX package or module set.
## Package 3: supported configurations and TLS

- [ ] Provide tested, minimal examples for extended health/readiness
endpoints, rate limits, connection limits, and ClickHouse HTTP proxying.
endpoints and ClickHouse HTTP proxying.
- [ ] Extend the qualified defaults to DNS resolution, upstream verification,
and the remaining profiles without silently breaking general-purpose use.
- [ ] Document configuration mounting, validation, reload, rollback, logging,
troubleshooting, and secret redaction.
- [ ] Implement and test structured logging for ClickHouse, extended health
endpoints, and request and connection limiting.
- [ ] Implement and test structured logging for ClickHouse and extended
health endpoints.
- [ ] Qualify runtime collection, rotation ownership, pipeline failure, and
retention evidence for the selected logging platform.
- [ ] Qualify lifecycle-alert delivery and exact platform cryptographic-policy
Expand Down
1 change: 1 addition & 0 deletions docs/SUPPORT.md
Original file line number Diff line number Diff line change
Expand Up @@ -29,6 +29,7 @@ Absence from a matrix means unqualified, not implicitly compatible.
| HTTP reverse-proxy profile | Preview/unqualified | Restricted-runtime, safe-header, logging, and upstream-failure tests exist; HTTPS upstreams and platform controls are outside this profile. |
| HTTP load-balancing profile | Preview/unqualified | Distribution, passive failure handling, bounded retries, and failover logging are tested on native AMD64/ARM64 Podman and Docker compatibility; capacity, latency, draining, and affinity are not qualified. |
| WebSocket proxying profile | Preview/unqualified | Upgrade forwarding, derived connection disposition, `101` relay, and unaffected plain HTTP are tested; frame exchange, session duration, and concurrent session capacity are not. |
| Request and connection limiting profile | Preview/unqualified | Rate rejection, connection rejection, bandwidth pacing, and unlimited health checks are tested; workload tuning, zone sizing, and exhausted-zone behaviour are not. |
| TLS termination and mTLS profiles | Preview/unqualified | TLS 1.2/1.3, client authentication, leaf renewal, CRL enforcement, and negative cases are tested; production PKI operations and exact-host cryptographic policy remain unqualified. |
| Verified HTTPS upstream profile | Preview/unqualified | Chain, hostname, SNI, revocation, overlapping-CA rotation, and restricted-runtime behavior are tested; deployment DNS, egress, and PKI remain operator-owned. |
| Linux AMD64 and ARM64 | Preview/unqualified | Native CI exists; release-candidate evidence is not complete. |
Expand Down
6 changes: 3 additions & 3 deletions docs/USE-CASES.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,9 +7,9 @@ negative cases, runtime restrictions, and operational guidance are tested
against the released image.

The development image has tested preview profiles for static content, HTTP and
verified-HTTPS upstream proxying, HTTP load balancing, WebSocket proxying, TLS
termination, and mutual TLS, including health endpoints and structured logging
on unprivileged ports. The other
verified-HTTPS upstream proxying, HTTP load balancing, WebSocket proxying,
request and connection limiting, TLS termination, and mutual TLS, including
health endpoints and structured logging on unprivileged ports. The other
profiles below are design targets for the first release unless stated
otherwise. See [Qualified HTTP and TLS configuration profiles](CONFIGURATION-PROFILES.md)
for the exact implemented boundary.
Expand Down
144 changes: 144 additions & 0 deletions examples/profiles/rate-limited/nginx.conf
Original file line number Diff line number Diff line change
@@ -0,0 +1,144 @@
worker_processes auto;
pid /tmp/nginx.pid;
error_log /dev/stderr notice;

events {
worker_connections 1024;
}

http {
include /etc/nginx/mime.types;
default_type application/octet-stream;

map $http_x_request_id $correlation_id {
"~^[A-Za-z0-9][A-Za-z0-9._-]{0,63}$" $http_x_request_id;
default $request_id;
}

# A connection that never produced a request has no method: a failed TLS
# handshake, a malformed request line, or a client that disconnects before
# the request is read. Those are not application requests, so they stay in
# the error stream instead of emitting a structured access event with empty
# fields.
map $request_method $is_application_request {
default 1;
"" 0;
}

# A location without a limit leaves these variables empty. Naming that case
# keeps every field in the access event non-empty, so "no value" and "the
# limit did not apply here" stay distinguishable to a reader.
map $limit_req_status $limit_req_result {
default $limit_req_status;
"" NOT_EVALUATED;
}

map $limit_conn_status $limit_conn_result {
default $limit_conn_status;
"" NOT_EVALUATED;
}

# The limit key is the direct peer address.
#
# Behind a load balancer or ingress controller that address is the proxy,
# so every client shares one bucket and the limit becomes a global cap
# rather than a per-client one. A deployment in that position needs a
# reviewed key derived from a forwarded address it actually trusts.
#
# Never key a limit on a header the client can set. A client that chooses
# its own key gets a fresh bucket per request and the limit stops existing.
# `realip` with a trusted-proxy list, or an equivalent reviewed mechanism,
# is what makes a forwarded address usable as a key.
#
# The zones are shared across workers. 10m holds roughly 160k IPv4 states;
# size them for the expected distinct-client count, because an exhausted
# zone fails closed and rejects new clients.
limit_req_zone $binary_remote_addr zone=requests_per_client:10m rate=10r/s;
limit_conn_zone $binary_remote_addr zone=connections_per_client:10m;

# 429 tells a client it was rate limited. The NGINX default is 503, which
# is indistinguishable from an outage and invites clients to retry harder.
limit_req_status 429;
limit_conn_status 429;
limit_req_log_level warn;
limit_conn_log_level warn;

log_format profile_json escape=json
'{"timestamp":"$time_iso8601",'
'"request_id":"$correlation_id",'
'"method":"$request_method",'
'"uri":"$uri",'
'"protocol":"$server_protocol",'
'"status":$status,'
'"body_bytes_sent":$body_bytes_sent,'
'"request_time":$request_time,'
'"limit_req_result":"$limit_req_result",'
'"limit_conn_result":"$limit_conn_result"}';
access_log /dev/stdout profile_json if=$is_application_request;

client_body_temp_path /tmp/nginx-client-body;
proxy_temp_path /tmp/nginx-proxy;
fastcgi_temp_path /tmp/nginx-fastcgi;
uwsgi_temp_path /tmp/nginx-uwsgi;
scgi_temp_path /tmp/nginx-scgi;

server_tokens off;
client_max_body_size 1m;
client_body_timeout 10s;
client_header_timeout 10s;
keepalive_timeout 30s;
send_timeout 30s;

server {
listen 8080 default_server;
listen [::]:8080 default_server;
server_name _;

root /srv/www;
autoindex off;

add_header X-Content-Type-Options nosniff always;
add_header X-Request-ID $correlation_id always;

# The health endpoint is deliberately outside both limits. A limited
# health check turns a traffic spike into a failed liveness probe and
# a restart, which removes capacity exactly when it is needed.
location = /healthz {
access_log off;
default_type text/plain;
return 200 "ok\n";
}

location / {
# `burst` absorbs normal bunching; `nodelay` serves the burst
# immediately instead of queueing it, so a well-behaved client is
# not slowed down while an abusive one is still capped. Without
# `nodelay` the burst is spread at the configured rate, which looks
# like latency to the caller.
limit_req zone=requests_per_client burst=20 nodelay;

# Concurrency is a separate budget from rate. A handful of slow
# downloads can hold every worker connection without ever
# exceeding the request rate.
limit_conn connections_per_client 10;

# Per-connection bandwidth is the third budget. Without it a
# single client can hold a connection inside the connection limit
# while consuming the link, and the server cannot reclaim the
# socket because it has already handed the body to the kernel.
#
# `limit_rate_after` leaves small responses untouched, so ordinary
# pages are unaffected and only long transfers are paced.
limit_rate_after 1m;
limit_rate 512k;

if ($request_method !~ ^(GET|HEAD)$) {
return 405;
}

location ~ (?:^|/)\. {
return 403;
}
}
}
}
Loading
Loading