feat(validate): add --queries flag for fleet-wide CQL parsing (closes #11)#13
Merged
Conversation
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…yntax Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Remove the stale local copy of QueryValidationResult from plan_formatter.py (which lacked the new `location` field) and replace it with an import of the canonical dataclass from deployment_orchestrator. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Adds --queries / -Q to talonctl validate. Schema validation runs offline as before; when the flag is set, credentials are required and every extracted query is CQL-parsed via the NGSIEM API. Schema errors short- circuit before any API calls are made. Credential errors surface a clear message pointing to talonctl auth setup. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…estrator Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…idate-query Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Closes #11. Closes the gap between
talonctl validate(schema-only) andtalonctl validate-query(one template at a time) by adding a CI-friendly fleet-wide CQL parser, and cleans up misleading"Unknown error"messaging inNGSIEMClient.talonctl validate --queries / -Qflag: after schema validation passes, parses every CQL query across detections, saved searches, and dashboards against NGSIEM in parallel. Requires credentials; hard-fails with a friendly message if missing.talonctl.core.query_collectionmodule (QueryRefdataclass +collect_queries_from_templatesdispatcher) that centralises per-resource-type query-field knowledge.DeploymentOrchestrator.validate_queries()reuses the existingThreadPoolExecutorpattern.QueryValidationResultgrew an optionallocationfield so output anchors at the exact widget / filter path (e.g.at: widgets.top_ips.queryString).NGSIEMClient.test_query_syntaxandexecute_queryno longer emit"Unknown error"fallbacks. Errors now readLogScale rejected query (status=<N>, no detail returned by API)when the upstream API returns nothing structured, or pass payload through verbatim when present.filter > query) aligned across plan path,validate-query, and the new collector; formatter wording neutralised away from FQL/deployment framing; dead"Unknown error"fallback removed fromvalidate-query.Plan/apply-path detection query validation (
_validate_detection_queries) is intentionally unchanged — broadening it to saved searches and dashboards is a tracked follow-up.Test plan
Offline (run in CI):
pytest tests/ -q→ 636 passedruff check src/ tests/→ cleanNGSIEMClienterror-message honesty (list/string/empty payloads);validate_queries()all-valid / one-invalid / per-query-exception / empty-fleet / filter-propagation; CLI--queriesshort-circuit on schema errors, credentials-missing path via realload_credentialsstub,-Qshort flag, location rendering in output.Live API (to run on a work machine before merging — see local
docs/superpowers/plans/2026-04-21-validate-cql-parity-live-tests.md):talonctl validate --querieson a clean fleet → exit 0!test(field, /x/i)) → exit 1 withat: search.filterat: widgets.<id>.queryStringat: queryString-Qshort flag paritybody.errorspayload shape (record a sample if the API ever does return detail)