Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
20 commits
Select commit Hold shift + click to select a range
f194d7a
test: stage adaptive SOC orchestration contract
seonghobae Aug 16, 2026
11506af
feat: stage adaptive SOC orchestration default
seonghobae Aug 16, 2026
bc50579
ci: apply and verify wardnet adaptive default
seonghobae Aug 16, 2026
eb61643
feat(ai): delegate SOC analysis to adaptive orchestration
github-actions[bot] Aug 16, 2026
4250362
test(ai): scope the auto-orchestration contract to SOC payload
seonghobae Aug 18, 2026
def964b
docs(ai): define the adaptive SOC orchestration boundary
seonghobae Aug 18, 2026
9174a72
build(deps): bump github/codeql-action/upload-sarif to 4.37.6 (#73)
dependabot[bot] Aug 20, 2026
8c8d4d5
Merge remote-tracking branch 'origin/main' into fix/pr76-rebase
seonghobae Aug 22, 2026
1cc4927
fix: address PR review follow-ups on the adaptive orchestrator default
seonghobae Aug 22, 2026
3f25ea5
build(deps): bump github/codeql-action/upload-sarif (#92)
dependabot[bot] Aug 26, 2026
43f373f
Merge branch 'main' into agent/adaptive-orchestrator-default
seonghobae Aug 26, 2026
edcb1a6
build(deps): bump futures-util from 0.3.33 to 0.3.34 (#91)
dependabot[bot] Aug 26, 2026
a4b85a5
Merge branch 'main' into agent/adaptive-orchestrator-default
opencode-agent[bot] Aug 26, 2026
1071176
Merge pull request #76 from ContextualWisdomLab/agent/adaptive-orches…
seonghobae Aug 26, 2026
c6501e9
fix(deploy): disable service account token automount (KSV-0036)
claude Aug 30, 2026
be45e69
fix(shutdown): register SIGTERM handler before announcing readiness
claude Aug 30, 2026
e67a939
fix(shutdown): register Windows Ctrl-C handler synchronously too
claude Aug 30, 2026
8319bed
fix(deploy): disable service account token automount (KSV-0036) (#132)
seonghobae Aug 30, 2026
b2bcee3
feat(threat-intel): recognize the CISA Known Exploited Vulnerabilitie…
seonghobae Aug 31, 2026
cc15cc2
fix(deploy): remove distributable administrator credential (#137)
seonghobae Sep 1, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .github/workflows/scorecard-analysis.yml
Original file line number Diff line number Diff line change
Expand Up @@ -29,6 +29,6 @@ jobs:
results_format: sarif
publish_results: false
- name: Upload SARIF
uses: github/codeql-action/upload-sarif@d1ba80a13dd99fba24a470575428917156a28b43 # v4
uses: github/codeql-action/upload-sarif@ff2f1c621b7f889edc0d3c761ac2e6a3f8cdb0dd # v4
with:
sarif_file: results.sarif
12 changes: 12 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
# Changelog

## Unreleased

### Security

- Removed the distributable Kubernetes administrator `Secret` and historical placeholder credential. Production deployments must provision `waf-ids-ai-soc-admin` / `ADMIN_TOKEN` through the external secret-management control plane; the workload's `secretKeyRef` is explicitly non-optional.
- Added a structural regression contract that rejects shipped administrator Secret objects, placeholder credentials, decoy workloads, init-container false positives, and optional administrator Secret references.

### Operations

- Documented administrator credential provisioning, rotation, rollout verification, rollback, evidence handling, and the boundary with the separate runtime-authentication fail-closed work tracked in issue #78.
2 changes: 1 addition & 1 deletion CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -62,7 +62,7 @@ The core stays an in-repo workspace crate on purpose (no git submodule) until it

## Runtime Configuration

Read in `run_from_env` (`src/lib.rs`): `BIND_ADDR` (default `127.0.0.1:8080`), `ADMIN_TOKEN` (write token for `X-Admin-Token`), `ADMIN_TOKENS` (comma-separated `token:actor` pairs for multi-token RBAC with per-token audit actors), `WAF_IDS_STATE_PATH` (optional JSON state file; omitted = seeded in-memory state), `DNSBL_ORIGIN` (default `dnsbl.local`), `EVENT_LIMIT` (default 1000, must be > 0), `RATE_LIMIT` / `RATE_LIMIT_WINDOW`.
Read in `run_from_env` (`src/lib.rs`): `BIND_ADDR` (default `127.0.0.1:8080`), `WAF_IDS_STATE_PATH` (optional JSON state file; omitted = seeded in-memory state), `DNSBL_ORIGIN` (default `dnsbl.local`), `EVENT_LIMIT` (default 1000, must be > 0), `RATE_LIMIT` / `RATE_LIMIT_WINDOW`, `WAF_IDS_CREDENTIALS_PATH` (optional JSON bootstrap file for process-local credentials/config), `ADMIN_TOKEN` (bootstrap transport for the shared write token), and `ADMIN_TOKENS` (bootstrap transport for comma-separated `token:actor[:role]` RBAC entries). `ADMIN_TOKEN` and `ADMIN_TOKENS` are loaded into `CredentialRegistry` before the server starts; handlers read the in-process registry/AppState copy, not raw env vars. KEV imports use the built-in CISA endpoint at runtime; only in-crate tests can override it through `AppState::with_kev_catalog_url` to point at a loopback mock server.

## Key Conventions

Expand Down
26 changes: 13 additions & 13 deletions Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

86 changes: 86 additions & 0 deletions crates/waf-ids-core/src/lib.rs
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,8 @@ pub const TARGET_SALE_VALUE_KRW: u64 = 2_000_000_000;
pub struct AppData {
pub routes: Vec<RouteConfig>,
pub threats: Vec<ThreatIndicator>,
#[serde(default)]
pub operator_threat_keys: Vec<ThreatIndicatorKey>,
Comment on lines +14 to +15

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔴 Upgrades lose existing threat ownership

Legacy state loads both ownership lists empty. Feed refreshes leave withdrawn entries active and can overwrite or later delete manually managed entries.

Prompt for agents
Existing persisted AppData has threats and threat_feeds but lacks operator_threat_keys and threat_feed_ownership. Serde defaults both new fields to empty, so ownership cannot be inferred after upgrade. The first refresh records only the current snapshot and cannot remove already-withdrawn legacy feed threats. Existing operator-created threats are also unmarked, allowing a matching feed import to overwrite their payload and a later refresh to delete them. Add an explicit persisted-state migration or a backward-compatible ownership model that preserves legacy operator data and safely establishes feed ownership before enabling destructive reconciliation. Cover loading a pre-change state file followed by multiple feed refreshes.
Devin Review

Was this helpful? React with 👍 or 👎 to provide feedback.

pub dnsbl: Vec<DnsblEntry>,
pub events: Vec<SecurityEvent>,
pub next_event_id: u64,
Expand All @@ -22,6 +24,8 @@ pub struct AppData {
pub commercial: CommercialProfile,
#[serde(default)]
pub threat_feeds: Vec<ThreatFeedStatus>,
#[serde(default)]
pub threat_feed_ownership: Vec<ThreatFeedOwnership>,
}

impl AppData {
Expand All @@ -42,6 +46,7 @@ impl AppData {
source: "seed:owasp-crs-shape".to_string(),
ttl_seconds: 86_400,
}],
operator_threat_keys: Vec::new(),
dnsbl: vec![DnsblEntry {
address: "203.0.113.10".parse().expect("seed IP address is valid"),
code: "127.0.0.2".to_string(),
Expand All @@ -56,6 +61,7 @@ impl AppData {
next_audit_log_id: 1,
commercial: CommercialProfile::seeded(),
threat_feeds: Vec::new(),
threat_feed_ownership: Vec::new(),
}
}
}
Expand Down Expand Up @@ -179,6 +185,19 @@ pub struct ThreatFeedStatus {
pub ttl_seconds: u64,
}

#[derive(Debug, Clone, Serialize, Deserialize, PartialEq, Eq)]
pub struct ThreatFeedOwnership {
pub feed_id: String,
pub threat_keys: Vec<ThreatIndicatorKey>,
}

#[derive(Debug, Clone, Serialize, Deserialize, PartialEq, Eq, Hash)]
pub struct ThreatIndicatorKey {
pub indicator_type: String,
pub value: String,
pub source: String,
}

#[derive(Debug, Clone, Serialize, Deserialize, PartialEq, Eq)]
pub struct ThreatFeedImport {
pub feed_id: String,
Expand Down Expand Up @@ -533,6 +552,32 @@ pub fn upsert_threat_feed(
feed
}

pub fn threat_indicator_key(indicator: &ThreatIndicator) -> ThreatIndicatorKey {
ThreatIndicatorKey {
indicator_type: indicator.indicator_type.clone(),
value: indicator.value.clone(),
source: indicator.source.clone(),
}
}

pub fn replace_threat_feed_ownership(
ownership: &mut Vec<ThreatFeedOwnership>,
feed_id: String,
threat_keys: Vec<ThreatIndicatorKey>,
) -> Vec<ThreatIndicatorKey> {
if let Some(existing) = ownership.iter_mut().find(|item| item.feed_id == feed_id) {
let previous = existing.threat_keys.clone();
existing.threat_keys = threat_keys;
previous
} else {
ownership.push(ThreatFeedOwnership {
feed_id,
threat_keys,
});
Vec::new()
}
}

pub fn record_audit_log(data: &mut AppData, entry: NewAuditLogEntry) -> AuditLogEntry {
let audit_log = AuditLogEntry {
id: data.next_audit_log_id,
Expand Down Expand Up @@ -845,6 +890,14 @@ pub fn score_request(
let kind = indicator.indicator_type.to_ascii_lowercase();
let matched = if matches!(kind.as_str(), "ip" | "client_ip" | "source_ip" | "src_ip") {
client_ip.is_some_and(|ip| indicator.value.parse::<IpAddr>().ok() == Some(ip))
} else if kind == "cve" {
// A CVE identifier is vulnerability-catalog metadata (e.g. from a
// CISA KEV import), not a request-content observable: it can
// legitimately appear in a vulnerability-management or security
// tool's own traffic (`/api/cve/CVE-2021-44228`), so it must never
// drive content-substring scoring. It stays visible via the
// threat-indicator, feed-freshness, and buyer-evidence APIs.
false
Comment on lines +893 to +900

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

📝 Info: CVE metadata stays outside enforcement

score_request excludes cve indicators from content matching. KEV catalog entries remain evidence metadata without blocking requests that mention a CVE.

Devin Review

Was this helpful? React with 👍 or 👎 to provide feedback.

} else {
haystack.contains(&indicator.value.to_lowercase())
};
Expand Down Expand Up @@ -1292,6 +1345,14 @@ fn buyer_evidence_endpoints() -> Vec<BuyerEvidenceEndpoint> {
"OpenCTI observable/indicator JSON ingest into threat indicators and DNSBL (admin-auth)",
false,
),
buyer_evidence_endpoint(
"cisa_kev_ingest",
"POST",
"/api/threat-intel/cisa-kev",
"application/json",
"CISA Known Exploited Vulnerabilities catalog pull into CVE threat indicators (admin-auth)",
false,
),
]
}

Expand Down Expand Up @@ -1441,6 +1502,31 @@ mod tests {
assert_eq!(miss.score, 0);
}

#[test]
fn score_request_never_content_matches_cve_indicators() {
// A CVE indicator (e.g. from a CISA KEV import) is vulnerability
// metadata, not a request-content signature: a security-tooling
// request can legitimately carry the literal CVE string, and that
// must never contribute to the block score.
let threats = vec![ThreatIndicator {
value: "CVE-2021-44228".to_string(),
indicator_type: "cve".to_string(),
severity: Severity::Critical,
source: "feed:cisa-kev".to_string(),
ttl_seconds: 86_400,
}];
let hit = score_request(
"/api/cve/CVE-2021-44228",
None,
"looking up CVE-2021-44228 details",
None,
&threats,
&[],
);
assert_eq!(hit.score, 0);
assert_eq!(hit.reason, "no matching indicator");
}

#[test]
fn score_request_saturates_instead_of_overflowing_on_many_matches() {
// Regression: `score` is a u16 accumulator. With enough matching
Expand Down
17 changes: 8 additions & 9 deletions deploy/kubernetes/waf-ids-ai-soc.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -3,15 +3,12 @@ kind: Namespace
metadata:
name: waf-ids-ai-soc
---
apiVersion: v1
kind: Secret
metadata:
name: waf-ids-ai-soc-admin
namespace: waf-ids-ai-soc
type: Opaque
stringData:
ADMIN_TOKEN: replace-with-secret-manager-sync
---
# The administrator Secret is intentionally not distributed with Wardnet.
# For a fresh install, create this Namespace idempotently first, provision
# `waf-ids-ai-soc-admin` through the organization's secret-management control
# plane, wait for synchronization, then apply this complete manifest. The
# Deployment has no fallback value and therefore fails closed when the Secret
# or `ADMIN_TOKEN` key is absent.
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
Expand Down Expand Up @@ -41,6 +38,7 @@ spec:
labels:
app.kubernetes.io/name: waf-ids-ai-soc
spec:
automountServiceAccountToken: false
securityContext:
runAsNonRoot: true
fsGroup: 10001
Expand All @@ -67,6 +65,7 @@ spec:
secretKeyRef:
name: waf-ids-ai-soc-admin
key: ADMIN_TOKEN
optional: false
volumeMounts:
- name: state
mountPath: /var/lib/waf-ids-ai-soc
Expand Down
90 changes: 90 additions & 0 deletions docs/adr/0010-adaptive-contextual-orchestrator-default.md

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

📝 Info: Research attachment rule is satisfied

The PR attaches permitted research PDFs and links restricted sources without redistribution. The new KEV and orchestration work follows the repository rule.

Devin Review

Was this helpful? React with 👍 or 👎 to provide feedback.

Original file line number Diff line number Diff line change
@@ -0,0 +1,90 @@
# ADR-0010: SOC analysis delegates default execution to contextual-orchestrator auto

- Status: Accepted
- Date: 2026-08-16

## Context

Wardnet's optional SOC analysis endpoint calls the organization LLM gateway, but a
model/messages-only request leaves policy implicit and can collapse into a fixed
single-worker path. Security analysis ranges from bounded enrichment to high-risk,
multi-step investigation. The edge gateway must not hard-code one provider, one
model, one topology, or one reasoning budget for every incident.

The orchestration literature also distinguishes delegation policy from the workers
that perform the task. TRINITY assigns Thinker, Worker, and Verifier roles turn by
turn; Conductor learns communication topologies and targeted worker instructions;
and Fugu adapts the resulting scaffold to the query. These results support an
adaptive default, but they do not authorize the orchestrator to own Wardnet's
security evidence, permissions, audit trail, or operator decision.

## Literature-to-decision mapping

| Source | Mechanism | Reported ablation/metric | Decision item it grounds |
| --- | --- | --- | --- |
| Xu et al. (2025), *TRINITY* | A lightweight (~0.6B parameter) coordinator assigns Thinker/Worker/Verifier roles turn-by-turn without modifying constituent model weights. | 86.2% pass@1 on LiveCodeBench at publication; the ablation attributes the gain to the coordinator's hidden-state contextualization versus RL/imitation-learning coordinators under the same budget. | Decision item 2 (role-separated worker and verifier execution) -- justifies delegating *when* to add an independent checking role to the orchestrator rather than hard-coding it in Wardnet. |
| Nielsen et al. (2025), *Conductor* | A 7B RL-trained coordinator adapts topology to task difficulty: single-query for factual tasks, planner-executor-verifier pipelines for hard tasks, with "Recursive Test-Time Scaling" (the coordinator can select itself as a worker for self-correction). | Record-setting 83.9% LiveCodeBench and 87.5% GPQA-Diamond; measured cost-efficiency gains over Mixture-of-Agents baselines. | Decision item 3 (conducted/recursive workflow for complex investigations) -- the query-adaptive depth and recursive self-correction are exactly the "decomposition, additional evidence, or iterative verification" case this item reserves for the orchestrator, not Wardnet. |
| Tang et al. (2026), *Sakana Fugu* | Fugu/Fugu-Ultra devise dynamic agentic scaffolds (large-scale fine-tuning + evolutionary search + RL) that vary with the query, orchestrating a team of diverse SOTA models. | State-of-the-art on SWE-Bench Pro, Terminal-Bench, LiveCodeBench, GPQA-Diamond, and Humanity's Last Exam; training explicitly optimizes the performance/latency trade-off via evolutionary search. | Decision item 1 (a quality-sufficient single route for bounded, low-ambiguity enrichment) and the "quality and safety take precedence over latency" rule -- query-adaptive scaffolding is the mechanism that keeps bounded events on a cheap route without Wardnet pre-selecting depth. |
| Omidvar & Akhlaghi (2026) | Models LLM sampling as a discrete stochastic channel, unifying retry/majority-vote/self-consistency into six operators, plus a cost-aware semantic-nearest-neighbor router with a single Lagrangian parameter traversing the quality-cost Pareto frontier. | ~56% lower normalized cost at matched quality, and ~7% quality improvement at matched cost (26% over single-shot), on MMLU/GSM8K/HumanEval. | The cost/quality tie-breaking rule ("known cost may break ties only after capability and safety constraints") -- grounds treating cost as a Pareto-frontier parameter subordinate to quality/safety, not an independent optimization target. |

The scheduled evaluations required by the Decision section (comparing single-route,
worker-verifier, and deeper orchestration modes on grounded SOC outcomes) are the
Wardnet-side analogue of each paper's own ablation methodology, applied to this
system's incident corpus instead of LiveCodeBench/GPQA/MMLU.

## Decision

The SOC request explicitly includes `orchestration_mode: "auto"`. The central
orchestrator may select, within its published contract:

1. a quality-sufficient single route for bounded, low-ambiguity enrichment;
2. role-separated worker and verifier execution when independent checking is
warranted; or
3. a conducted or recursive workflow for complex investigations that require
decomposition, additional evidence, or iterative verification.

Role-specific reasoning effort, workflow depth, tool access, and stopping policy are
owned by contextual-orchestrator. Quality and safety requirements take precedence
over latency. Known cost may break ties only after capability and safety constraints;
unpriced providers are not treated as free. Scheduled evaluations must compare the
single-route, worker-verifier, and deeper orchestration modes so that increased
inference depth is retained only when it improves grounded SOC outcomes.

Wardnet retains WAF/IDS evidence collection, authorization, bounded request handling,
audit records, security-domain validation, and operator presentation. The
orchestrator receives only the bounded event representation constructed by Wardnet,
and its response remains untrusted analysis until Wardnet validates and presents it.
Explicit fixed modes remain controlled experiments and rollback controls, not the
product default. Unsupported orchestration capabilities or malformed responses fail
closed rather than silently changing the security decision path.

## Consequences

- Adding or removing worker models does not change the Wardnet API contract.
- The gateway continues to make enforcement decisions from deterministic Wardnet
evidence; LLM output cannot directly block, allow, mutate policy, or obtain an
administrative capability.
- Audit and evaluation evidence must identify the selected orchestration mode and
model roles without copying secrets or unrelated personal data.
- A future live-model test lane must measure grounding, unsupported-claim rate,
decision consistency, and the marginal benefit of deeper orchestration on a fixed
incident corpus.

## References

Nielsen, S., Cetin, E., Schwendeman, P., Sun, Q., Xu, J., & Tang, Y. (2025).
*Learning to orchestrate agents in natural language with the Conductor* [Preprint].
arXiv. https://doi.org/10.48550/arXiv.2512.04388

Omidvar, H., & Akhlaghi, V. (2026). *A communication-theoretic framework for LLM
agents: Cost-aware adaptive reliability* [Preprint]. arXiv.
https://doi.org/10.48550/arXiv.2605.09121

Tang, Y., Cetin, E., Xu, J., Sun, Q., Nielsen, S., Richard, V., Goda, H., Tymchenko,
I., Nguyen, N., Lee, H., Ashiga, M., Kotyan, S., Kuroki, S., & Clanuwat, T. (2026).
*Sakana Fugu technical report* [Technical report]. arXiv.
https://doi.org/10.48550/arXiv.2606.21228

Xu, J., Sun, Q., Schwendeman, P., Nielsen, S., Cetin, E., & Tang, Y. (2025).
*TRINITY: An evolved LLM coordinator* [Preprint]. arXiv.
https://doi.org/10.48550/arXiv.2512.04695
Loading
Loading