Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,3 +10,4 @@
### Operations

- Documented administrator credential provisioning, rotation, rollout verification, rollback, evidence handling, and the boundary with the separate runtime-authentication fail-closed work tracked in issue #78.
- Added buyer-facing readiness gauges to `GET /metrics`, including enterprise sale readiness, readiness check pass/fail counts, enabled-route readiness, and admin-auth configuration.
8 changes: 7 additions & 1 deletion docs/analytics/soc-kpis.md
Original file line number Diff line number Diff line change
Expand Up @@ -37,4 +37,10 @@

## MVP Measurement

The baseline exposes `GET /api/kpis` with counts for routes, indicators, DNSBL entries, threat feeds, fresh feeds, stale feeds, events, blocked events, monitored events, and management audit logs. `GET /api/commercial/evidence-manifest` adds the buyer-facing checklist that maps those signals to required runtime endpoints, committed documents, and deployment assets. Latency, precision, triage time, and full feed freshness percentages require the next telemetry and analyst-disposition work.
The baseline exposes `GET /api/kpis` with counts for routes, indicators, DNSBL entries, threat feeds, fresh feeds, stale feeds, events, blocked events, monitored events, and management audit logs. `GET /api/commercial/evidence-manifest` adds the buyer-facing checklist that maps those signals to required runtime endpoints, committed documents, and deployment assets. `GET /metrics` now exports those gauges plus buyer-meaningful operational readiness signals: enterprise sale readiness, readiness check pass/fail counts, gateway route readiness, and whether admin write authentication is configured. These counters and readiness gauges provide the continuous, broadly available operational signals advocated by [Sigelman et al.](https://research.google.com/pubs/pub36356.html), but they do not yet provide the request-level traces needed to diagnose latency across components. Latency, precision, triage time, and full feed freshness percentages still require the next telemetry and analyst-disposition work; in particular, [Chandola et al.](https://doi.org/10.1145/1541880.1541882) show that anomaly-detection methods have different trade-offs, so detection quality must be validated with precision and false-positive measurements rather than inferred from event volume alone.

## Research Grounding

- **Sigelman et al. (2010), [*Dapper, a Large-Scale Distributed Systems Tracing Infrastructure*](https://research.google.com/pubs/pub36356.html).** Production tracing benefits from low-overhead, ubiquitous instrumentation and continuous monitoring. This supports collecting gateway latency and readiness signals continuously, while also motivating future request-level traces for cross-component diagnosis.
- **Xu et al. (2009), [*Detecting Large-Scale System Problems by Mining Console Logs*](https://doi.org/10.1145/1629575.1629587).** Structured features derived from operational logs can expose runtime problems and produce operator-facing explanations. This grounds retaining event, feed-error, and triage signals as inputs to future anomaly detection rather than treating raw log volume as a detection result.
- **Chandola, Banerjee, and Kumar (2009), [*Anomaly Detection: A Survey*](https://doi.org/10.1145/1541880.1541882).** Anomaly-detection techniques vary by data assumptions, computational cost, and application context. This supports pairing detection counts with analyst-confirmed precision, false-positive rate, and time-to-triage guardrails.
3 changes: 3 additions & 0 deletions docs/commercial/buyer-due-diligence.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,7 @@
- Security events: `GET /api/events`
- SOC event export: `GET /api/events.ndjson`
- SOC KPIs: `GET /api/kpis`
- Prometheus operations metrics: `GET /metrics`
- License profile: `GET /api/commercial/license`
- Sale readiness: `GET /api/commercial/readiness`
- Buyer evidence manifest: `GET /api/commercial/evidence-manifest`
Expand All @@ -27,6 +28,7 @@
- Authenticated management writes through `X-Admin-Token`.
- Automated tests for management APIs, gateway scoring, DNSBL export, event NDJSON export, feed freshness, persistence failures, commercial readiness, and legacy state compatibility.
- Buyer evidence manifest that lists required runtime endpoints, committed document paths, deployment assets, blockers, and runtime evidence counts from one API.
- Prometheus metrics that expose readiness blockers, passing/failing readiness checks, route readiness, and admin-auth configuration without scraping the admin console.
- `scripts/smoke.sh` verifies a full local lifecycle including restart persistence.

## Security Review Packet
Expand Down Expand Up @@ -58,6 +60,7 @@ Then inspect:
```bash
curl -fsS http://127.0.0.1:8080/api/commercial/readiness
curl -fsS http://127.0.0.1:8080/api/commercial/evidence-manifest
curl -fsS http://127.0.0.1:8080/metrics
curl -fsS http://127.0.0.1:8080/api/threat-feeds/freshness
curl -fsS http://127.0.0.1:8080/api/events.ndjson
curl -fsS http://127.0.0.1:8080/api/support-bundle
Expand Down
113 changes: 111 additions & 2 deletions src/lib.rs
Original file line number Diff line number Diff line change
Expand Up @@ -282,7 +282,7 @@ impl AppState {
dnsbl_origin: self.dnsbl_origin.clone(),
event_limit: self.event_limit,
credentials_source: self.credentials_source.as_str().to_string(),
admin_auth_configured: self.admin_token.is_some() || !self.admin_tokens.is_empty(),
admin_auth_configured: has_write_admin_credential(self),
Comment thread
seonghobae marked this conversation as resolved.
}
}
}
Expand Down Expand Up @@ -1104,7 +1104,7 @@ async fn evaluate_request(
async fn metrics(State(state): State<AppState>) -> impl IntoResponse {
let body = {
let data = state.inner.read().await;
prometheus_exposition(&kpi_snapshot_at(&data, now_unix()))
operational_prometheus_exposition(&state, &data, now_unix())
};
(
[(
Expand All @@ -1115,6 +1115,72 @@ async fn metrics(State(state): State<AppState>) -> impl IntoResponse {
)
}

fn operational_prometheus_exposition(state: &AppState, data: &AppData, now_unix: u64) -> String {
let kpis = kpi_snapshot_at(data, now_unix);
let readiness = commercial_readiness_snapshot_at(data, now_unix);
let routes_enabled = data.routes.iter().filter(|route| route.enabled).count();

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

📝 Info: Readiness gauges preserve endpoint semantics

Both route gauges use the same enabled-route predicate as readyz. One state snapshot keeps their values mutually consistent.

Devin Review

Was this helpful? React with 👍 or 👎 to provide feedback.

let checks_passed = readiness
.checks
.iter()
.filter(|check| check.status == ReadinessStatus::Pass)
.count();
let checks_failed = readiness
.checks
.iter()
.filter(|check| check.status == ReadinessStatus::Fail)
.count();
let mut out = prometheus_exposition(&kpis);
append_prometheus_gauge(
&mut out,
"waf_ids_commercial_ready",
"Enterprise sale readiness flag (1=ready, 0=blocked).",
usize::from(readiness.ready_for_enterprise_sale),
);
append_prometheus_gauge(
&mut out,
"waf_ids_commercial_blockers",
"Current count of commercial-readiness blockers.",
readiness.blockers.len(),
);
append_prometheus_gauge(
&mut out,
"waf_ids_readiness_checks_passed",
"Commercial-readiness checks currently passing.",
checks_passed,
);
append_prometheus_gauge(
&mut out,
"waf_ids_readiness_checks_failed",
"Commercial-readiness checks currently failing.",
checks_failed,
);
append_prometheus_gauge(
&mut out,
"waf_ids_gateway_ready",
"Kubernetes readiness-style route readiness (1=ready, 0=not ready).",
usize::from(routes_enabled > 0),
);
append_prometheus_gauge(
&mut out,
"waf_ids_gateway_routes_enabled",
"Enabled gateway route count used by the readiness probe.",
routes_enabled,
);
append_prometheus_gauge(
&mut out,
"waf_ids_admin_auth_configured",
"Admin write authentication configured (1=write-capable credential configured, 0=no write-capable credential configured; readonly auth may still exist).",
usize::from(has_write_admin_credential(state)),
Comment thread
devin-ai-integration[bot] marked this conversation as resolved.
Comment thread
seonghobae marked this conversation as resolved.
);
out
}

fn append_prometheus_gauge(out: &mut String, name: &str, help: &str, value: usize) {
out.push_str(&format!(
"# HELP {name} {help}\n# TYPE {name} gauge\n{name} {value}\n"
));
}

async fn get_commercial_license(State(state): State<AppState>) -> Json<CommercialProfile> {
Json(state.inner.read().await.commercial.clone())
}
Expand Down Expand Up @@ -3707,6 +3773,41 @@ mod tests {
);
let body = body_text(response).await;
assert!(body.contains("waf_ids_security_events_blocked"));
assert!(body.contains("waf_ids_commercial_ready 0"));
assert!(body.contains("waf_ids_commercial_blockers 4"));
assert!(body.contains("waf_ids_readiness_checks_passed 2"));
assert!(body.contains("waf_ids_readiness_checks_failed 4"));
assert!(body.contains("waf_ids_gateway_ready 1"));
assert!(body.contains("waf_ids_gateway_routes_enabled 1"));
assert!(body.contains("waf_ids_admin_auth_configured 0"));
}

#[tokio::test]
async fn admin_auth_metric_requires_a_write_capable_credential() {
let readonly_app = build_app(
AppState::seeded(None).with_admin_tokens(parse_admin_tokens("read:auditor:readonly")),
);
let readonly_health: HealthStatus =
json_body(app_request(&readonly_app, empty_request(Method::GET, "/healthz")).await)
.await;
assert!(!readonly_health.admin_auth_configured);
let readonly_body =
body_text(app_request(&readonly_app, empty_request(Method::GET, "/metrics")).await)
.await;
assert!(readonly_body.contains("waf_ids_admin_auth_configured 0"));
assert!(readonly_body.contains(
"# HELP waf_ids_admin_auth_configured Admin write authentication configured (1=write-capable credential configured, 0=no write-capable credential configured; readonly auth may still exist)."
));

let writer_app = build_app(
AppState::seeded(None).with_admin_tokens(parse_admin_tokens("write:operator:write")),
);
let writer_health: HealthStatus =
json_body(app_request(&writer_app, empty_request(Method::GET, "/healthz")).await).await;
assert!(writer_health.admin_auth_configured);
let writer_body =
body_text(app_request(&writer_app, empty_request(Method::GET, "/metrics")).await).await;
assert!(writer_body.contains("waf_ids_admin_auth_configured 1"));
}

#[tokio::test]
Expand Down Expand Up @@ -7645,6 +7746,14 @@ mod tests {
let health = authed.health_status();
assert_eq!(health.credentials_source, "file");
assert!(health.admin_auth_configured);

let readonly = state
.clone()
.with_admin_tokens(parse_admin_tokens("tok:auditor:readonly"))
.with_credentials_source(CredentialSource::File);
let health = readonly.health_status();
assert_eq!(health.credentials_source, "file");
assert!(!health.admin_auth_configured);
}

fn clearfolio_test_config(base_url: &str) -> ClearfolioConfig {
Expand Down
Loading