Repository navigation
feat: add a qualified preview ClickHouse HTTP proxying profile - #22
Merged
Merged
Conversation
Bound the shape of a request to the ClickHouse HTTP interface: methods, body size, timeouts, retries, and temporary storage. State plainly that this proxy is not an authorization boundary. NGINX parses HTTP, not SQL, so it cannot make a session read-only, restrict which statements run, bound returned rows, or tell a SELECT from a DROP. Those remain ClickHouse `readonly` settings, quotas, row policies, and grants. A reader who expects the proxy to constrain query content should find that stated rather than infer it. This is where the repository-wide logging rule earns its keep. The interface accepts `?query=` and often `?user=&password=`, so the request target routinely carries credentials and customer data. Logging `$uri` rather than the request line makes both structurally absent instead of filtered, and the test asserts a password and a column name sent as parameters appear nowhere in the container output. Verified against a config logging `$request_uri`, where the assertion fails as it should. `proxy_max_temp_file_size 0` with buffering off is a direct consequence of two earlier decisions: a read-only root and a 64 MB `/tmp` tmpfs mean one large result set could otherwise fill the tmpfs and take the container down. Two findings while running it: - A request the proxy refuses never reaches an upstream, so every upstream variable is empty and the schema rejected the event. Those cases now record NONE, keeping "the database rejected this" distinguishable from "the database never saw it". - The log validator required every scenario to be a GET, which was a leftover from when they all were. The expected method is now explicit, defaulting to GET so existing scenarios keep asserting what they asserted. The fixture stands in for the HTTP interface and executes nothing, so interoperation with a real ClickHouse server is explicitly not qualified. Testing against a real server image would mean pulling a large external image into CI, against the hermetic no-pull posture the build now holds.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Last of the Package 3 example profiles. Preview/unqualified.
This proxy is not an authorization boundary
Stated prominently in the config and the profile doc, because it is the thing
most likely to be assumed wrongly.
NGINX parses HTTP, not SQL. It cannot make a session read-only, restrict which
statements run, bound how many rows a query returns, or tell a
SELECTfrom aDROP. Those belong to ClickHouse:readonlysettings, quotas, row policies,per-user grants. This profile bounds the shape of a request — method, body
size, timeouts, temporary storage — not its meaning.
Where the logging rule earns its keep
The ClickHouse HTTP interface accepts
?query=...and, commonly,?user=...&password=.... The request target therefore routinely carriescredentials and customer data.
Every profile in this repo logs
$uriand never$request,$request_uri, or$args, so those are structurally absent rather than filtered. The test issuesa request carrying a password and a column name as parameters and requires that
neither string appears anywhere in the container output.
I verified the assertion has teeth by pointing the log format at
$request_uri: it fails withforbidden value occurred in logs, as it should.The profile doc flags the corollary — a downstream collector, error tracker, or
front proxy that records full request lines reintroduces exactly what this
removes.
Temporary storage
proxy_max_temp_file_size 0with buffering off is a direct consequence of twoearlier decisions in this repo. A result set larger than the proxy buffers
would spill to
proxy_temp_path, which sits on the container's 64 MB/tmptmpfs under a read-only root. One large result could fill the tmpfs and take
the container down, so the response is streamed instead.
Two findings from running it
Requests the proxy refuses have no upstream. The
403and413casesproduced empty upstream fields and the schema rejected them. They now record
NONE, which keeps "the database rejected this" distinguishable from "thedatabase never saw it" — a gateway problem versus a query problem.
The validator required every scenario to be a GET, a leftover from when
they all were; this profile's query is a POST. The expected method is now
explicit and defaults to GET, so every existing scenario keeps asserting
exactly what it asserted before.
Three consecutive green local runs on rootless Podman.
Scope, and the fixture decision
The fixture stands in for the HTTP interface — it reports what the proxy
forwarded and can answer with an exception code. It executes nothing and
implements no part of the ClickHouse protocol.
Testing against a real
clickhouse-serverimage would mean pulling a largeexternal image into CI, against the hermetic no-pull posture established in
#13. So interoperation with a real server, protocol and version behaviour,
query semantics, compression negotiation, and performance against real result
sets are explicitly not qualified, in the profile doc and the support
matrix. A deployment qualifies those against its own server.
Roadmap
Closes both remaining ClickHouse items. Package 3 drops from six open items to
four: DNS and upstream-verification defaults, the configuration-operations
doc, and the two blocked on a chosen logging platform and an exact supported
host.
🤖 Generated with Claude Code