fix: correct camelCase/snake_case param mismatch causing Attack Discovery to report 0 discoveries - #52
Merged
maxcold merged 6 commits intoJul 8, 2026
Conversation
…very to report 0 discoveries (elastic#46) Kibana's anonymization-fields _find endpoint only recognizes snake_case `per_page`; the client sent camelCase `perPage`, which Zod silently dropped, falling back to the default page size of 20. That truncated field list starved the LLM prompt of key ECS fields (host.name, user.name, rule.name, ...), causing it to hallucinate alert IDs that Kibana's post-processing then filtered out — surfacing as "0 discoveries" with no error anywhere in the chain. - src/elastic/client/attackDiscoveryClient.ts: use per_page (snake_case) - src/elastic/service/attackDiscoveryService.ts: fail-loud guard that warns when the anonymization field list looks truncated (<=20 fields or missing key ECS fields) so a repeat of this regresses loudly instead of silently - src/elastic/service/rulesService.ts, casesService.ts: schema-source comments citing the verified Kibana Zod schema for every hand-built param object, after auditing all other API calls for the same mistake - CONTRIBUTING.md: new convention — verify param casing against the Kibana schema source, don't assume, and cite the schema file inline - scripts/e2e/attackDiscoveryPoll.ts, scripts/e2e/run.ts, scripts/repro/attack-discovery-race.ts: shared generation-poll/ connector-selection helper, wired into an e2e smoke test that generates a discovery and asserts count > 0 — this is the regression test for elastic#46 (fails loudly if the param mismatch reappears, skips cleanly with no LLM connector configured)
Discovery count depends on a live LLM's non-deterministic output, so asserting discoveries > 0 is a useful local/manual regression check for elastic#46 but too flaky to gate an unattended CI/treadmill run on. Detect CI via CI=true/CI=1 and skip by default there; MCP_E2E_ATTACK_DISCOVERY_SKIP still overrides explicitly (0 forces it on, 1 forces it off) in either environment.
…per-page-param-46
- Rename shared-cluster credentials path from ~/.treadmill/mcp-e2e-clusters.json to ~/.mcp-e2e/clusters.json and drop the operator-specific env var alias - Trim verbose comments in the e2e/repro harness and attack discovery service/client down to the essentials - Add scripts/**/*.ts to eslint.config.js so scripts/ is actually linted - Gitignore .treadmill/ and context/ (local operator scaffolding, never meant to be committed)
…n only Scopes the PR down to the elastic#46 fix itself: the per_page param correction, the fail-loud truncation guard, the schema-source comments on the other Kibana API calls audited for the same mistake, and the CONTRIBUTING.md convention. The live e2e/evals harness and the repro script were useful for diagnosis but add scope (new scripts, new package.json entries, a new docs file) beyond what this bug fix needs.
Debug logging was scaffolding for the elastic#46 repro; the fail-loud truncation guard is the durable prevention mechanism and needs no env flag. .treadmill/ and context/ entries were leftover from local tooling, not part of this fix.
Collaborator
Author
|
This is ready for review — validated end-to-end against a live Kibana/ES cluster (real `generate-attack-discovery` run against seeded data, confirmed `discoveries > 0` and the new fail-loud guard stays silent), plus CI green. @maxcold @davethegut — tagging you both since you're on issue #46. As an external contributor I don't have permission to request reviews directly, so flagging here instead. |
maxcold
self-requested a review
July 8, 2026 08:17
maxcold
approved these changes
Jul 8, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Fixes #46. Attack Discovery intermittently reported 0 discoveries with no error surfaced anywhere in the chain.
Root cause: the anonymization-fields
_findendpoint (/api/security_ai_assistant/anonymization_fields/_find) only recognizes the snake_case query paramper_page. The client sent camelCaseperPage, which Kibana's Zod schema silently dropped, falling back to the default page size of 20. That truncated field list starved the LLM prompt of key ECS fields (host.name,user.name,rule.name, ...), causing it to hallucinate alert IDs that Kibana's post-processing then filtered out — surfacing as "0 discoveries" with no error anywhere in the chain.Fix: send
per_page(snake_case) inattackDiscoveryClient.ts.Prevention (since this class of bug is easy to reintroduce):
attackDiscoveryService.tsnow warns if the anonymization field list looks truncated (≤20 fields, or missing key ECS fields) — so a repeat of this regression is loud, not silent.rulesService.ts,casesService.ts) for the same camelCase/snake_case mistake and added schema-source comments citing the verified Kibana Zod schema for each.CONTRIBUTING.mdconvention: verify param casing against the Kibana schema source before assuming, and cite the schema file inline.Changes
src/elastic/client/attackDiscoveryClient.ts—per_page(snake_case), wasperPagesrc/elastic/client/attackDiscoveryClient.test.ts— updated assertionsrc/elastic/service/attackDiscoveryService.ts— fail-loud truncation guardsrc/elastic/service/rulesService.ts,src/elastic/service/casesService.ts— schema-source comments after auditCONTRIBUTING.md— new param-casing convention.gitignore— minor local-tooling ignoresTest plan
npx vitest runon affected service/client tests — passingnpm run typecheck— cleannpm run lint— cleanperPage→ Zod drop → 20-field truncation → hallucinated alert IDs → 0 discoveries) against a local Kibana dev cell with seeded sample data and a live LLM connector, and confirmed the fix resolves it (generation succeeded with discoveries > 0, fail-loud guard stayed silent).