Feat: Rule suppression - #40
Conversation
Shadow mode names the rule that would have blocked. Naming it is half an answer: without a way to say "that one is wrong for this form", the operator chooses between promoting with the false positives and not promoting at all. That is the choice that stops WAFs being turned on. A suppression says a detection does not count for a scope. It hooks into execute_defense_node, which every mechanism returns through, so it applies uniformly rather than needing support in each one. The safety property that shapes the design: a mechanism reports one aggregate score for all its flags, so there is no per-flag score to subtract. A node is therefore neutralised only when *every* one of its flags was suppressed. One surviving flag means something nobody suppressed still fired, and that verdict stands. Verified live: with kw:viagra suppressed, a viagra payload goes 403 -> 200, a casino payload stays 403, and viagra+casino stays 403. Scopes accumulate -- global, vhost and endpoint all apply -- rather than the narrower replacing the broader. That is the opposite of config_resolver's deep merge, deliberately: an endpoint entry silently dropping the global ones is the wrong default for a control that makes the WAF do less. A global "*" is refused. It would be a WAF that is running and does nothing, which is too easy to reach by a single typo. Suppressions are logged when applied, bounded at 200 since the list is walked on every defense node, and never expire from cache -- an expiring suppression would silently start blocking traffic an operator deliberately allowed. UI: a Suppressions page under Security, and a Suppress action on each detection in the shadow view that carries the flag and vhost across. The recorded flag is profile-prefixed (legacy:kw:viagra) while a suppression matches what the mechanism emits (kw:viagra), so the prefix is stripped on the way. The shortcut only appears when every recorded decision came from one vhost -- otherwise the prefilled scope would be a guess, and a suppression on the wrong vhost is a hole rather than an annoyance. RBAC: admin and operator create/read/delete, viewer read. Verified live. Gates: 73 unit specs (16 new), config contracts hold, contract check 12 endpoints, integration 42 passed / 2 known gaps / 0 failed, typecheck 0 errors, lint 0 errors / 59 warnings.
The chart does not mount the image's nginx.conf, it ships its own copy in a ConfigMap. Anything added to one and not the other fails silently, and only on Kubernetes -- never in the compose stack, which is where testing happens. Two live consequences. Shadow mode does not work on Kubernetes at all. lua_shared_dict shadow_cache was never added to the chart, so ngx.shared.shadow_cache is nil, and shadow_recorder guards on that and returns early. No crash, no log, just an empty sample forever. That is my regression, merged in PR #39. HAPROXY_TIMEOUT is set by the openresty deployment but was not declared, so os.getenv returns nil and the value an operator puts in values.yaml is ignored. This is the same variable whose regression was fixed in PR #36 -- the fix works under compose and has been dead on Kubernetes throughout. Also declares the eight other variables nginx.conf knows and the chart did not: REDIS_TLS, WAF_ADMIN_COOKIE, WAF_ADMIN_URL, WAF_ALLOW_INTERNAL_URLS, WAF_DISABLE_SSRF_PROTECTION, WAF_LOCAL_AUTH, WAF_LOG_HMAC_KEY, WAF_SESSION_TTL and WAF_TRUSTED_PROXIES. None are set by the chart today so nothing is currently broken by them, but without the declaration an operator cannot make them work via extraEnv either. WAF_TRUSTED_PROXIES is the one that matters: it drives F01 client-IP extraction, in exactly the ingress topology where getting the client IP right is hardest. scripts/check-helm-drift.py compares the two files and fails on any difference in either direction, and runs in CI. Verified it catches drift by deleting a dict and confirming a non-zero exit.
There was a problem hiding this comment.
Pull request overview
Adds a rule-suppression feature end-to-end (OpenResty runtime + Redis sync + Admin API + Admin UI), letting operators mark specific detection flags as “not counting” for a given scope without disabling an entire endpoint/vhost.
Changes:
- Implements suppression storage/sync (Redis hash → shared dict) and applies suppressions during defense-node execution.
- Adds admin API endpoints + RBAC permissions for managing suppressions, with UI pages and Shadow Mode shortcuts.
- Adds CI guard to detect drift between image
nginx.confand Helm chart’s embedded copy (env +lua_shared_dict).
Reviewed changes
Copilot reviewed 19 out of 19 changed files in this pull request and generated 7 comments.
Show a summary per file
| File | Description |
|---|---|
| scripts/check-helm-drift.py | New CI script to compare env/shared-dict declarations between image and Helm chart config. |
| openresty/spec/suppressions_spec.lua | Unit specs covering matching, scoping, and application semantics. |
| openresty/lua/suppressions.lua | Suppression matching/scoping logic and application to defense-node results. |
| openresty/lua/redis_sync.lua | Synces suppression data from Redis into a dedicated shared dict cache. |
| openresty/lua/rbac.lua | Adds suppressions permissions and routes to the RBAC map. |
| openresty/lua/defense_profile_executor.lua | Hooks suppression application into the defense-node execution path. |
| openresty/lua/api_handlers/suppressions.lua | New API handler for listing/creating/deleting suppressions. |
| openresty/lua/admin_api.lua | Registers suppressions handler and routes DELETE /suppressions/{id}. |
| openresty/conf/nginx.conf | Adds lua_shared_dict suppression_cache to the image config. |
| helm/forms-waf/templates/openresty-configmap.yaml | Syncs env + shared dict declarations in the chart’s nginx.conf copy. |
| docs/openapi.yaml | Documents suppression schema and /suppressions endpoints. |
| docs/API_HANDLERS.md | Adds handler documentation and RBAC resource/action list for suppressions. |
| admin-ui/src/pages/shadow/ShadowMode.tsx | Adds “Suppress” shortcut from Shadow Mode counts. |
| admin-ui/src/pages/security/Suppressions.tsx | New UI for creating/listing/removing suppressions. |
| admin-ui/src/components/layout/Sidebar.tsx | Adds Suppressions entry under Security navigation. |
| admin-ui/src/App.tsx | Adds route for /security/suppressions. |
| admin-ui/src/api/generated.ts | Generated OpenAPI bindings updated with suppressions types/paths. |
| admin-ui/src/api/client.ts | Adds suppressionsApi client and related TS types. |
| .github/workflows/quality.yml | Runs the new Helm/nginx.conf drift check in CI. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| function detectionFlag(recorded: string): string { | ||
| const firstColon = recorded.indexOf(':') | ||
| return firstColon === -1 ? recorded : recorded.slice(firstColon + 1) | ||
| } |
| ngx.log(ngx.INFO, "SUPPRESSED: ", defense_name, " flags=", | ||
| table.concat(removed, ","), " vhost=", tostring(request_context.vhost_id), | ||
| " endpoint=", tostring(request_context.endpoint_id)) |
| local encoded = cjson.encode(entry) | ||
| local ok, set_err = red:hset(REDIS_KEY, id, encoded) | ||
| utils.close_redis(red) | ||
| if not ok then | ||
| return utils.error_response("Failed to store suppression: " .. (set_err or "unknown")) | ||
| end | ||
|
|
||
| -- Push it to this pod's workers now; other pods pick it up on their timer. | ||
| redis_sync.sync_now() | ||
|
|
||
| ngx.log(ngx.WARN, "SUPPRESSION_ADDED: flag=", flag, " scope=", scope_type, | ||
| ":", tostring(scope_id), " by=", entry.created_by) | ||
|
|
||
| return utils.json_response({ | ||
| suppression = entry, | ||
| created = not already, | ||
| }, already and 200 or 201) | ||
| end |
| --- Every suppression currently in force, as stored by redis_sync. | ||
| -- @return array of {id, scope_type, scope_id, flag, reason, created_at, created_by} | ||
| function _M.get_all() | ||
| local dict = cache() | ||
| if not dict then return {} end | ||
|
|
||
| local raw = dict:get(CACHE_KEY) | ||
| if not raw then return {} end | ||
|
|
||
| local decoded = cjson.decode(raw) | ||
| if type(decoded) ~= "table" then return {} end | ||
| return decoded | ||
| end |
| env HAPROXY_TIMEOUT; | ||
| env REDIS_TLS; | ||
| env WAF_ADMIN_COOKIE; | ||
| env WAF_ADMIN_URL; | ||
| env WAF_ALLOW_INTERNAL_URLS; |
| post: | ||
| summary: Stop a detection counting for one scope | ||
| responses: | ||
| "201": | ||
| description: Created |
| delete: | ||
| summary: Remove every suppression | ||
| responses: | ||
| "200": | ||
| description: Cleared | ||
|
|
Seven findings, all valid. Two of them are holes in work from this same
branch that I had reported as verified.
The drift checker had a blind spot. Its env regex was [A-Z_]-only, so it
printed "env declarations: identical" while nginx.conf's three lowercase
proxy declarations -- http_proxy, https_proxy, no_proxy -- were missing from
the Helm ConfigMap. A gate that reports success over real drift is worse than
no gate. Regex widened to accept either case, the three declarations added,
and the count went 26 -> 29.
detectionFlag() mangled unprefixed flags. It stripped everything up to the
first colon, turning kw:viagra into viagra, which matches no suppression.
Flags are usually profile-prefixed, but defense-line flags are merged into
the result without a prefix, so both that rule and the suggested "strip only
if the remainder still has a colon" are wrong -- the latter on single-segment
flags like legacy:thread_error. The decision is now made against the profile
names the API already reports in top_rules: a leading segment is stripped
only when it is one of them.
The suppression audit log was invisible. It was at INFO while the default
error_log level is warn, so "logged, not silent" was not true. Raising every
hit to WARN would be its own problem -- a suppression exists because its rule
fires often. So apply() now reports whether it actually changed the verdict,
and only that case logs at WARN: "SUPPRESSED (block prevented)". Dropping a
flag from a node that was not going to block stays at INFO. Verified the WARN
line appears at the default level.
Idempotency now comes from HSET, not a best-effort scan. HSET returns 1 for a
created field and 0 for a replaced one; the earlier read_all() scan could
misreport under a concurrent write. The scan remains only for the 200-entry
cap. Verified 201 then 200, with one entry stored.
get_all() decoded the same JSON on every defense node. Now cached per worker
and keyed on the raw blob, so a redis_sync write is picked up on the next
call rather than after a TTL -- a stale suppression is either traffic wrongly
blocked or wrongly allowed, and neither should wait. A spec drives three
successive config changes to prove the cache turns over.
OpenAPI was under-specified: POST /suppressions had no requestBody (generated
types marked it never) and omitted the 200 idempotent response, DELETE
/suppressions documented no body while returning JSON, and DELETE
/suppressions/{id} was missing entirely despite existing in the router and in
RBAC.
Gates: 75 unit specs, helm drift clean at 29/29 and 14/14, config contracts
hold, contract check 12 endpoints, integration 42 passed / 2 known gaps / 0
failed, typecheck 0 errors, lint 0 errors / 59 warnings.
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 19 out of 19 changed files in this pull request and generated no new comments.
Suppressed comments (5)
openresty/lua/api_handlers/suppressions.lua:121
- POST /suppressions only checks that scope_id is non-empty for vhost/endpoint scopes; a typo yields a suppression that never applies, which is hard to diagnose (and contradicts RBAC’s comment that scope is enforced on creation). Consider validating that the referenced vhost/endpoint config exists before accepting the suppression.
local red, conn_err = utils.get_redis()
if not red then
return utils.error_response("Redis connection failed: " .. (conn_err or "unknown"))
end
local existing = read_all(red)
openresty/lua/api_handlers/suppressions.lua:45
- read_all() accepts any JSON value from Redis; if a stored suppression value decodes to a non-table, later code assumes fields like .id/.created_at exist and may error. Only include decoded entries that are JSON objects (Lua tables).
This issue also appears on line 116 of the same file.
for i = 1, #raw, 2 do
local decoded = cjson.decode(raw[i + 1])
if decoded then
decoded.id = decoded.id or raw[i]
out[#out + 1] = decoded
end
end
openresty/lua/suppressions.lua:109
- active_patterns() assumes every entry from get_all() is a table; if the cache contains any non-table values (e.g. corrupted Redis data making it into the list), indexing entry.flag will raise and can break request handling. Add a type guard before reading fields.
for _, entry in ipairs(_M.get_all()) do
if entry.flag and applies_to_scope(entry, vhost_id, endpoint_id) then
patterns[#patterns + 1] = entry.flag
end
openresty/lua/redis_sync.lua:312
- sync_suppressions() appends any JSON value returned by cjson.decode(); if a hash field contains non-object JSON (string/number/bool), the suppressions list will contain non-tables and later code (suppressions.active_patterns) will error when it indexes entry.flag. Filter decoded values to tables only.
for i = 1, #entries, 2 do
local decoded = cjson.decode(entries[i + 1])
if decoded then
decoded.id = decoded.id or entries[i]
list[#list + 1] = decoded
end
end
admin-ui/src/pages/security/Suppressions.tsx:52
- scopeType is initialised by casting the query param to SuppressionScope; if someone lands on /security/suppressions?scope_type=foo (or a stale link), the state becomes an invalid value that won’t match the Select options and can break form logic. Guard the query param against the allowed literals before using it.
const [scopeType, setScopeType] = useState<SuppressionScope>(
(searchParams.get('scope_type') as SuppressionScope | null) ?? 'vhost'
)
const [scopeId, setScopeId] = useState(searchParams.get('scope_id') ?? '')
Copilot's second pass reported no new comments on the seven fixes and suppressed five further observations. Four are one class worth taking seriously, and one of those is a request-path fault rather than a nitpick. cjson.decode returns a number for "12345" and a string for a quoted scalar. Both are truthy, so `if decoded then` let them through, and every later access assumed a table. In active_patterns that access is `entry.flag`, which raises -- and the suppression hook sits *after* the pcall that wraps the mechanism, so the error does not degrade to a neutral result, it propagates into request handling. Anything written straight into the Redis hash that is not a JSON object was enough to do it. Guarded in all three places: redis_sync keeps a bad value out of the cache and logs it, the API handler skips it when listing, and active_patterns skips it in the request path so a single corrupt entry cannot take a request down. Verified by writing a string and a number directly into waf:suppressions: both were logged and ignored, a clean request stayed 200, a casino request stayed 403, and the one valid suppression kept applying. Scope ids are now checked against Redis on create. A typo was accepted and then quietly never matched anything -- config that looks applied and is inert, which is the exact failure this codebase has been bitten by repeatedly. A nonexistent vhost or endpoint now returns 400 saying so. The UI validated scope_type instead of casting it. A stale link carrying ?scope_type=foo put the form into a state no Select option matched, so the control looked set and would have submitted something else. Gates: 76 unit specs, helm drift clean, config contracts hold, contract check 12 endpoints, integration 42 passed / 2 known gaps / 0 failed, typecheck 0 errors, lint 0 errors / 59 warnings.
No description provided.