diff --git a/.claude-plugin/plugin.json b/.claude-plugin/plugin.json index 23e20fc..8431e91 100644 --- a/.claude-plugin/plugin.json +++ b/.claude-plugin/plugin.json @@ -1,8 +1,8 @@ { - "name": "complypack", + "name": "comply", "displayName": "ComplyPack", "version": "0.1.0", - "description": "Generate Rego policies from Gemara catalogs and extract assessment requirements via MCP server", + "description": "Gemara compliance pipeline and policy generation via MCP server", "author": { "name": "ComplyTime Authors", "url": "https://github.com/complytime" @@ -16,6 +16,8 @@ "opa", "gemara", "policy", - "mcp" + "mcp", + "audit", + "governance" ] } diff --git a/.cursor-plugin/plugin.json b/.cursor-plugin/plugin.json index bbc1c40..62c2af4 100644 --- a/.cursor-plugin/plugin.json +++ b/.cursor-plugin/plugin.json @@ -1,8 +1,8 @@ { - "name": "complypack", + "name": "comply", "displayName": "ComplyPack", "version": "0.1.0", - "description": "Generate Rego policies from Gemara catalogs and extract assessment requirements via MCP server", + "description": "Gemara compliance pipeline and policy generation via MCP server", "author": { "name": "ComplyTime Authors", "url": "https://github.com/complytime" @@ -15,6 +15,8 @@ "opa", "gemara", "policy", - "mcp" + "mcp", + "audit", + "governance" ] } diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index 7580aef..c0b3fa0 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -55,6 +55,7 @@ jobs: permissions: contents: read issues: read + pull-requests: read call_reusable_security: name: Security Scan diff --git a/.gitignore b/.gitignore index adff2f0..35db7f3 100644 --- a/.gitignore +++ b/.gitignore @@ -28,9 +28,11 @@ go.work.sum .DS_Store Thumbs.db -# Local docs (plans, analysis) +# Local docs (plans, analysis, specs, demos) docs/plans/ docs/analysis/ +docs/superpowers/ +docs/demo/ # Development tooling .opencode/ diff --git a/docs/adr/012-container-mcp-distribution.md b/docs/adr/012-container-mcp-distribution.md new file mode 100644 index 0000000..1606ac6 --- /dev/null +++ b/docs/adr/012-container-mcp-distribution.md @@ -0,0 +1,32 @@ +# ADR 012: Container-Based MCP Server Distribution + +**Status:** Proposed + +**Date:** 2026-06-08 + +**Context:** + +ComplyPack's MCP server is a Go binary that users must build locally. Issue #24 requires single-click distribution. Four options were evaluated: + +1. **Container image** — `docker run --rm -i ghcr.io/complytime/complypack` +2. **Binary download via plugin SessionStart hook** — download pre-built binaries from GitHub Releases into `~/.claude/plugins/data/` +3. **`go install`** — users install via Go toolchain +4. **Homebrew tap** — formula for macOS/Linux + +Option 2 was rejected on supply chain grounds: no plugin in the Claude Code ecosystem ships unsigned binaries, and downloading executables into the plugin data directory has no verification standard. The MCP security surface is already a known concern (CVE-2025-59536, CVE-2026-21852). + +Option 3 requires the Go toolchain on every user's machine — a non-starter for non-developer users. + +Option 4 adds a distribution channel to maintain and doesn't cover Fedora users natively. + +**Decision:** + +Distribute the MCP server as a multi-arch container image (`ghcr.io/complytime/complypack`). Sign images with cosign (keyless/OIDC). Users invoke via `docker run --rm -i` or `podman run --rm -i` in their `.mcp.json`. + +**Consequences:** + +- Users must have Docker or Podman installed — acceptable for the target audience (macOS and Fedora developers) +- Container startup adds ~1-2s latency on first invocation (image pull is one-time) +- OCI registry authentication for pulling Gemara catalogs works from inside the container (standard Docker credential chain) +- Image size will be ~30-50MB (Go static binary in distroless/alpine) +- No unsigned binary downloads, no Go toolchain requirement diff --git a/docs/adr/013-cli-flags-mcp-serve.md b/docs/adr/013-cli-flags-mcp-serve.md new file mode 100644 index 0000000..e6474d1 --- /dev/null +++ b/docs/adr/013-cli-flags-mcp-serve.md @@ -0,0 +1,36 @@ +# ADR 013: CLI Flags for `mcp serve` Configuration + +**Status:** Proposed + +**Date:** 2026-06-08 + +**Context:** + +`mcp serve` requires a `complypack.yaml` config file. In a containerized deployment, getting a config file into the container requires a volume mount (`-v ./complypack.yaml:/config/complypack.yaml:ro`), which adds friction — the file must exist at a known path before the MCP server starts. + +The MCP server only uses two config sections: `gemara.sources` (which OCI artifacts to load) and `schemas` (which platform schemas to serve). Fields like `id`, `evaluator-id`, `policies.dir`, and `output.dir` are only used by `pack` and `scan`. + +**Decision:** + +Add repeatable `--source` and `--schema` flags to `mcp serve`: + +```shell +complypack mcp serve \ + --source oci://registry.example.com/gemara/controls:v1 \ + --source oci+http://localhost:5001/gemara/guidance:v1 \ + --schema ci=cue://cue.dev/x/githubactions@v0#Workflow \ + --schema kubernetes +``` + +**`--source`** accepts `oci://` (TLS) or `oci+http://` (plain HTTP) URIs. The `+http` scheme variant provides per-source plain-HTTP control without a global flag. + +**`--schema`** accepts either a bare platform name (`kubernetes` — uses embedded schema) or `platform=source` syntax (`ci=cue://cue.dev/x/githubactions@v0#Workflow` — loads from the specified source). + +When `--source` flags are present, they replace the config file for source resolution. `--config` remains supported and takes precedence if both are provided. + +**Consequences:** + +- Containerized MCP servers need no volume mount — all configuration passes through `args` in `.mcp.json` +- Users edit one file (`.mcp.json`) instead of two (`.mcp.json` + `complypack.yaml`) +- `pack` and `scan` commands are unaffected — they continue to require `complypack.yaml` +- The `oci+http://` URI scheme is non-standard but self-documenting and avoids a global `--plain-http` flag that would apply to all sources diff --git a/docs/adr/014-parameter-delta-engine.md b/docs/adr/014-parameter-delta-engine.md new file mode 100644 index 0000000..43d8b15 --- /dev/null +++ b/docs/adr/014-parameter-delta-engine.md @@ -0,0 +1,36 @@ +# ADR 014: Parameter Delta Gathering Engine + +**Status:** Proposed + +**Date:** 2026-06-10 + +**Context:** + +Gemara Policies bind parameters at the org level as structured YAML (L3), while Guidance Catalogs (L1) and Control Catalogs (L2) express parameter expectations in prose — requirement text like "the system MUST require multi-factor authentication" or "builds SHOULD achieve at least SLSA Build Level 1." + +When preparing the mapping stage of the comply pipeline, the model needs to see L3 parameter values alongside the L1/L2 requirement text they map to. Without tooling, the model must make many MCP calls and manually cross-reference requirement IDs to build this picture. + +Three approaches were considered: + +1. **No tool** — the model reads policy and catalog resources via MCP and cross-references manually. Works but requires many calls and is error-prone for large catalogs. +2. **Heuristic comparison engine** — classify parameter specificity (concrete vs generic) via string matching, compute verdicts (aligned, mismatch, org_binds_generic). Rejected: heuristics for detecting generic language are brittle, and interpreting whether values differ meaningfully is what the model does well. +3. **Gathering engine** — walk the resolved policy graph, pair each structured L3 parameter with the L1/L2 requirement text it maps to, return them side by side. Let the model interpret the relationship. + +Option 3 was chosen. The engine handles what it's good at — traversing the resolved policy graph and collecting structured pairs. The model handles what it's good at — interpreting prose and judging parameter relationships. + +**Decision:** + +Implement a parameter gathering engine (`requirement.AnalyzeDelta`) that pairs L3 parameter values with L1/L2 requirement text across the resolved policy graph. Each pair contains: + +- `requirement_id` — which requirement the parameter maps to +- `label` — the parameter name +- `policy_value` — the structured value from the L3 Policy +- `requirement_text` — the prose from the L1/L2 catalog + +Expose this as the `analyze_parameter_delta` MCP tool so the `/comply` pipeline's mapping stage can read comparisons directly from the server. The mapping stage model interprets each pair in domain context and presents its assessment to the user. + +**Consequences:** + +- The mapping stage consumes structured pairs from MCP rather than manually cross-referencing artifacts +- Interpretation of parameter relationships (which is stricter, whether values conflict) is the model's responsibility — no heuristics +- The engine is intentionally simple: traverse the graph, collect pairs, return them diff --git a/docs/adr/015-comply-pipeline-plugin.md b/docs/adr/015-comply-pipeline-plugin.md new file mode 100644 index 0000000..b1720f3 --- /dev/null +++ b/docs/adr/015-comply-pipeline-plugin.md @@ -0,0 +1,39 @@ +# ADR 015: Comply Pipeline as Plugin Skills + +**Status:** Proposed + +**Date:** 2026-06-11 + +**Context:** + +ComplyPack's MCP server provides compliance data (controls, parameters, resolved policies) but has no opinion on workflow. Users need guided, multi-stage audit preparation: scoping which controls apply, mapping parameter deltas across frameworks, and producing an adherence plan (the applicability statement). + +Three approaches were considered: + +1. **Hardcoded CLI workflow** — `complypack comply --stage scoping`. Rigid; can't adapt to user context or partial completion. +2. **Autonomous agent loop** — a single agent prompt that runs all stages. Risk of generating control IDs or parameter values from model memory rather than source data when context windows grow large. +3. **Plugin skills with MCP grounding** — decompose the pipeline into discrete skills (`/comply:pipeline`, `/comply:pack`, `/comply:setup`), each reading from MCP resources at every step. + +Option 2 was rejected because the core safety property is that every stage must read control IDs, requirement IDs, and parameter values from MCP resources — never from model memory. A single long-running agent loop makes this harder to enforce. + +Option 1 was rejected because audit workflows are inherently conversational: auditors need to review scoping decisions, adjust parameter bindings, and approve the applicability statement before it's finalized. + +Option 3 was chosen. Skills provide structured guidance while keeping the human in the loop. MCP grounding ensures data integrity. The pipeline router checks `.complytime/` artifact state to resume from the correct stage. + +**Decision:** + +Implement the comply pipeline as plugin skills: + +- **`/comply:pipeline`** — router that inspects `.complytime/` directory state and dispatches to the correct stage (scoping → mapping → adherence) +- **`/comply:pack`** — generates assessment logic after pipeline completion +- **`/comply:setup`** — configures `.mcp.json` for the user's environment + +Each stage reads exclusively from MCP resources. The pipeline produces Gemara Policy artifacts. + +**Consequences:** + +- Users interact conversationally with each stage rather than running a batch process — audit decisions are reviewed before being recorded +- The pipeline is stateless across sessions: `.complytime/` artifacts are the checkpoint, not conversation history +- Adding new stages (e.g., evidence collection, continuous monitoring) means adding new skill files and updating the router +- The plugin registers with Claude Code, Cursor, and Gemini via their respective plugin manifests — same skills, three runtimes +- MCP grounding is a hard constraint: if the MCP server is unreachable, the pipeline fails rather than proceeding with stale or generated data diff --git a/gemini-extension.json b/gemini-extension.json index 554c82c..39a3b3d 100644 --- a/gemini-extension.json +++ b/gemini-extension.json @@ -1,5 +1,5 @@ { - "name": "complypack", - "description": "Generate Rego policies from Gemara catalogs and extract assessment requirements via MCP server", + "name": "comply", + "description": "Gemara compliance pipeline and policy generation via MCP server", "version": "0.1.0" } diff --git a/internal/mcp/consts.go b/internal/mcp/consts.go index 7ac6ebe..0671f58 100644 --- a/internal/mcp/consts.go +++ b/internal/mcp/consts.go @@ -8,6 +8,7 @@ const ( // Resource types ResourceTypeCatalog = "catalog" + ResourceTypeMapping = "mapping" ResourceTypeSchema = "schema" ResourceTypeEvaluator = "evaluator" diff --git a/internal/mcp/resources.go b/internal/mcp/resources.go index c0ec459..4b32ec5 100644 --- a/internal/mcp/resources.go +++ b/internal/mcp/resources.go @@ -12,6 +12,7 @@ import ( "github.com/complytime/complypack/internal/evaluator" "github.com/complytime/complypack/internal/requirement" "github.com/complytime/complypack/internal/schema" + "github.com/gemaraproj/go-gemara" "github.com/modelcontextprotocol/go-sdk/mcp" "gopkg.in/yaml.v3" ) @@ -56,10 +57,11 @@ func (rs *ResourceStore) CUESchema(platform string) (cue.Value, error) { func (rs *ResourceStore) ListResources(ctx context.Context) ([]mcp.Resource, error) { var resources []mcp.Resource - for name := range rs.artifacts { + for name, artifact := range rs.artifacts { + kind := artifactKind(artifact) resources = append(resources, mcp.Resource{ URI: fmt.Sprintf("%s://%s/%s", URIScheme, ResourceTypeCatalog, name), - Name: fmt.Sprintf("Gemara Artifact: %s", name), + Name: fmt.Sprintf("Gemara %s: %s", kind, name), MIMEType: MIMETypeYAML, }) } @@ -118,6 +120,24 @@ func (rs *ResourceStore) ReadResource(ctx context.Context, uri string) ([]*mcp.R Text: string(data), }}, nil + case ResourceTypeMapping: + if len(parts) != 2 { + return nil, fmt.Errorf("invalid URI format: %s", uri) + } + artifact, ok := rs.artifacts[parts[1]] + if !ok { + return nil, fmt.Errorf("mapping document %q not found", parts[1]) + } + data, err := yaml.Marshal(artifact) + if err != nil { + return nil, fmt.Errorf("failed to marshal mapping document %q: %w", parts[1], err) + } + return []*mcp.ResourceContents{{ + URI: uri, + MIMEType: MIMETypeYAML, + Text: string(data), + }}, nil + case ResourceTypeSchema: if len(parts) == 1 || parts[1] == "" { return rs.readSchemaListResource(uri) @@ -141,6 +161,21 @@ func (rs *ResourceStore) ReadResource(ctx context.Context, uri string) ([]*mcp.R } } +func artifactKind(artifact any) string { + switch artifact.(type) { + case *gemara.Policy: + return "Policy" + case *gemara.ControlCatalog: + return "ControlCatalog" + case *gemara.GuidanceCatalog: + return "GuidanceCatalog" + case *gemara.MappingDocument: + return "MappingDocument" + default: + return "Artifact" + } +} + func (rs *ResourceStore) readSchemaListResource(uri string) ([]*mcp.ResourceContents, error) { type schemaInfo struct { Platform string `json:"platform"` diff --git a/internal/mcp/server.go b/internal/mcp/server.go index 9a762db..5cc438f 100644 --- a/internal/mcp/server.go +++ b/internal/mcp/server.go @@ -113,6 +113,9 @@ func NewServer(ctx context.Context, opts *ServerOptions) (*Server, error) { for id, p := range loaded.Policies { allArtifacts[id] = p } + for id, md := range loaded.Mappings { + allArtifacts[id] = md + } // Load schemas from configured sources schemaReg := schema.DefaultRegistry() @@ -156,6 +159,17 @@ func NewServer(ctx context.Context, opts *ServerOptions) (*Server, error) { mcpServer.AddResource(resource, createResourceHandler(store, uri)) } + // Register mapping document resources + for name := range loaded.Mappings { + uri := fmt.Sprintf("%s://%s/%s", URIScheme, ResourceTypeMapping, name) + resource := &mcp.Resource{ + URI: uri, + Name: fmt.Sprintf("Gemara Mapping Document: %s", name), + MIMEType: MIMETypeYAML, + } + mcpServer.AddResource(resource, createResourceHandler(store, uri)) + } + // Register schema list resource (discovery) schemaListURI := fmt.Sprintf("%s://%s", URIScheme, ResourceTypeSchema) mcpServer.AddResource(&mcp.Resource{ @@ -198,6 +212,9 @@ func NewServer(ctx context.Context, opts *ServerOptions) (*Server, error) { assessmentTool := createGetAssessmentRequirementsTool() mcpServer.AddTool(assessmentTool, handleGetAssessmentRequirements(store)) + deltaTool := createAnalyzeParameterDeltaTool() + mcpServer.AddTool(deltaTool, handleAnalyzeParameterDelta(store, loaded)) + return &Server{ mcp: mcpServer, ResourceStore: store, diff --git a/internal/mcp/tool_assessment.go b/internal/mcp/tool_assessment.go index be6a28f..c8b2c60 100644 --- a/internal/mcp/tool_assessment.go +++ b/internal/mcp/tool_assessment.go @@ -9,6 +9,7 @@ import ( "strings" "github.com/complytime/complypack/internal/requirement" + "github.com/gemaraproj/go-gemara" "github.com/modelcontextprotocol/go-sdk/mcp" ) @@ -28,6 +29,13 @@ func createGetAssessmentRequirementsTool() *mcp.Tool { "type": "string", "description": "Optional: Specific control ID to filter requirements (e.g., 'CTRL-001')", }, + "scope": map[string]interface{}{ + "type": "array", + "items": map[string]interface{}{ + "type": "string", + }, + "description": "Optional: Filter requirements by applicability groups (e.g., ['maturity-1', 'maturity-2']). Returns requirements whose applicability contains any of the given values.", + }, }, "required": []interface{}{"catalogName"}, }, @@ -48,8 +56,9 @@ func handleGetAssessmentRequirements(store *ResourceStore) mcp.ToolHandler { return func(ctx context.Context, req *mcp.CallToolRequest) (*mcp.CallToolResult, error) { // Parse input var input struct { - CatalogName string `json:"catalogName"` - ControlID string `json:"controlId"` + CatalogName string `json:"catalogName"` + ControlID string `json:"controlId"` + Scope []string `json:"scope"` } if err := json.Unmarshal(req.Params.Arguments, &input); err != nil { @@ -58,14 +67,18 @@ func handleGetAssessmentRequirements(store *ResourceStore) mcp.ToolHandler { rp, found := store.resolved[input.CatalogName] if !found { - return nil, fmt.Errorf("policy %q not found", input.CatalogName) + rp, found = resolveFromCatalog(store, input.CatalogName) + if !found { + return nil, fmt.Errorf("policy or catalog %q not found", input.CatalogName) + } } - requirements := extractFromResolvedPolicy(rp, input.ControlID) + requirements := extractFromResolvedPolicy(rp, input.ControlID, input.Scope) // Build response responseData, err := json.Marshal(map[string]interface{}{ "catalog": input.CatalogName, "control_id": input.ControlID, + "scope": input.Scope, "count": len(requirements), "requirements": requirements, }) @@ -83,8 +96,41 @@ func handleGetAssessmentRequirements(store *ResourceStore) mcp.ToolHandler { } } +// resolveFromCatalog wraps a bare catalog in a synthetic ResolvedPolicy so +// get_assessment_requirements works with catalog names, not just policy names. +func resolveFromCatalog(store *ResourceStore, name string) (*requirement.ResolvedPolicy, bool) { + art, ok := store.artifacts[name] + if !ok { + return nil, false + } + cat, ok := art.(*gemara.ControlCatalog) + if !ok { + return nil, false + } + set := &requirement.ArtifactSet{ + Catalogs: map[string]*gemara.ControlCatalog{name: cat}, + Policies: make(map[string]*gemara.Policy), + Guidance: make(map[string]*gemara.GuidanceCatalog), + Mappings: make(map[string]*gemara.MappingDocument), + } + syntheticPolicy := gemara.Policy{ + Metadata: gemara.Metadata{ + Id: name + "-synthetic", + MappingReferences: []gemara.MappingReference{{Id: name}}, + }, + Imports: gemara.Imports{ + Catalogs: []gemara.CatalogImport{{ReferenceId: name}}, + }, + } + rp, err := requirement.ResolvePolicy(syntheticPolicy, set) + if err != nil { + return nil, false + } + return rp, true +} + // extractFromResolvedPolicy extracts requirements from a resolved policy graph. -func extractFromResolvedPolicy(rp *requirement.ResolvedPolicy, filterControlID string) []AssessmentRequirementInfo { +func extractFromResolvedPolicy(rp *requirement.ResolvedPolicy, filterControlID string, filterScope []string) []AssessmentRequirementInfo { var results []AssessmentRequirementInfo controlIDs := rp.ControlIDs() @@ -94,6 +140,9 @@ func extractFromResolvedPolicy(rp *requirement.ResolvedPolicy, filterControlID s for _, controlID := range controlIDs { for _, req := range rp.RequirementsForControl(controlID) { + if len(filterScope) > 0 && !applicabilityIntersects(req.Applicability, filterScope) { + continue + } info := AssessmentRequirementInfo{ ID: req.Id, ControlID: controlID, @@ -120,6 +169,17 @@ func extractFromResolvedPolicy(rp *requirement.ResolvedPolicy, filterControlID s return results } +func applicabilityIntersects(applicability, scope []string) bool { + for _, a := range applicability { + for _, s := range scope { + if a == s { + return true + } + } + } + return false +} + // GetAssessmentRequirementsHandler returns the handler (for testing). func GetAssessmentRequirementsHandler(store *ResourceStore) mcp.ToolHandler { return handleGetAssessmentRequirements(store) diff --git a/internal/mcp/tool_assessment_test.go b/internal/mcp/tool_assessment_test.go index 83f0b8a..6748042 100644 --- a/internal/mcp/tool_assessment_test.go +++ b/internal/mcp/tool_assessment_test.go @@ -24,11 +24,12 @@ func testResolvedPolicy() *requirement.ResolvedPolicy { { Id: "TEST-001-AR1", Text: "Test requirement", - Applicability: []string{"test"}, + Applicability: []string{"maturity-1", "maturity-2", "maturity-3"}, }, { - Id: "TEST-001-AR2", - Text: "Second requirement", + Id: "TEST-001-AR2", + Text: "Second requirement", + Applicability: []string{"maturity-2", "maturity-3"}, }, }, }, @@ -36,8 +37,9 @@ func testResolvedPolicy() *requirement.ResolvedPolicy { Id: "TEST-002", AssessmentRequirements: []gemara.AssessmentRequirement{ { - Id: "TEST-002-AR1", - Text: "Third requirement", + Id: "TEST-002-AR1", + Text: "Third requirement", + Applicability: []string{"maturity-3"}, }, }, }, @@ -125,6 +127,54 @@ func TestHandleGetAssessmentRequirements(t *testing.T) { assert.Len(t, requirements, 3) }) + t.Run("catalog name fallback", func(t *testing.T) { + catalog := &gemara.ControlCatalog{ + Metadata: gemara.Metadata{Id: "bare-catalog"}, + Controls: []gemara.Control{ + { + Id: "CAT-001", + AssessmentRequirements: []gemara.AssessmentRequirement{ + {Id: "CAT-001-AR1", Text: "Catalog requirement", Applicability: []string{"maturity-1"}}, + }, + }, + }, + } + catalogStore := &ResourceStore{ + artifacts: map[string]any{"bare-catalog": catalog}, + resolved: map[string]*requirement.ResolvedPolicy{}, + schemas: map[string][]byte{}, + } + catalogHandler := handleGetAssessmentRequirements(catalogStore) + + input := map[string]interface{}{ + "catalogName": "bare-catalog", + } + inputJSON, err := json.Marshal(input) + require.NoError(t, err) + + req := &mcp.CallToolRequest{ + Params: &mcp.CallToolParamsRaw{ + Arguments: json.RawMessage(inputJSON), + }, + } + + result, err := catalogHandler(context.Background(), req) + require.NoError(t, err) + require.NotNil(t, result) + + textContent, ok := result.Content[0].(*mcp.TextContent) + require.True(t, ok) + + var response map[string]interface{} + err = json.Unmarshal([]byte(textContent.Text), &response) + require.NoError(t, err) + + assert.Equal(t, float64(1), response["count"]) + requirements := response["requirements"].([]interface{}) + firstReq := requirements[0].(map[string]interface{}) + assert.Equal(t, "CAT-001-AR1", firstReq["id"]) + }) + t.Run("policy not found", func(t *testing.T) { input := map[string]interface{}{ "catalogName": "nonexistent", @@ -210,6 +260,31 @@ func TestHandleGetAssessmentRequirements(t *testing.T) { params := firstReq["parameters"].(map[string]interface{}) assert.Equal(t, "90", params["threshold"]) }) + + t.Run("filter by scope", func(t *testing.T) { + input := map[string]interface{}{ + "catalogName": "test-policy", + "scope": []string{"maturity-2"}, + } + inputJSON, err := json.Marshal(input) + require.NoError(t, err) + + req := &mcp.CallToolRequest{ + Params: &mcp.CallToolParamsRaw{ + Arguments: json.RawMessage(inputJSON), + }, + } + + result, err := handler(context.Background(), req) + require.NoError(t, err) + + textContent := result.Content[0].(*mcp.TextContent) + var response map[string]interface{} + err = json.Unmarshal([]byte(textContent.Text), &response) + require.NoError(t, err) + + assert.Equal(t, float64(2), response["count"]) + }) } func TestCreateGetAssessmentRequirementsTool(t *testing.T) { @@ -234,6 +309,10 @@ func TestCreateGetAssessmentRequirementsTool(t *testing.T) { require.True(t, ok) assert.Equal(t, "string", controlId["type"]) + scope, ok := properties["scope"].(map[string]interface{}) + require.True(t, ok) + assert.Equal(t, "array", scope["type"]) + required, ok := schema["required"].([]interface{}) require.True(t, ok) assert.Contains(t, required, "catalogName") @@ -243,20 +322,48 @@ func TestExtractFromResolvedPolicy(t *testing.T) { rp := testResolvedPolicy() t.Run("extract all", func(t *testing.T) { - results := extractFromResolvedPolicy(rp, "") + results := extractFromResolvedPolicy(rp, "", nil) assert.Len(t, results, 3) }) t.Run("filter by control", func(t *testing.T) { - results := extractFromResolvedPolicy(rp, "TEST-001") + results := extractFromResolvedPolicy(rp, "TEST-001", nil) assert.Len(t, results, 2) assert.Equal(t, "TEST-001", results[0].ControlID) assert.Equal(t, "TEST-001", results[1].ControlID) }) t.Run("parameters populated from assessment plans", func(t *testing.T) { - results := extractFromResolvedPolicy(rp, "TEST-001") + results := extractFromResolvedPolicy(rp, "TEST-001", nil) assert.Equal(t, "90", results[0].Parameters["threshold"]) assert.Empty(t, results[1].Parameters) }) + + t.Run("filter by scope", func(t *testing.T) { + results := extractFromResolvedPolicy(rp, "", []string{"maturity-2"}) + assert.Len(t, results, 2) + for _, r := range results { + assert.Contains(t, r.Applicability, "maturity-2") + } + }) + + t.Run("filter by multiple scope values", func(t *testing.T) { + results := extractFromResolvedPolicy(rp, "", []string{"maturity-1", "maturity-3"}) + assert.Len(t, results, 3) + }) + + t.Run("filter by scope and control", func(t *testing.T) { + results := extractFromResolvedPolicy(rp, "TEST-001", []string{"maturity-2"}) + assert.Len(t, results, 2) + }) + + t.Run("scope filters out non-matching", func(t *testing.T) { + results := extractFromResolvedPolicy(rp, "", []string{"maturity-1"}) + assert.Len(t, results, 1) + }) + + t.Run("nil scope returns all", func(t *testing.T) { + results := extractFromResolvedPolicy(rp, "", nil) + assert.Len(t, results, 3) + }) } diff --git a/internal/mcp/tool_delta.go b/internal/mcp/tool_delta.go new file mode 100644 index 0000000..82ee722 --- /dev/null +++ b/internal/mcp/tool_delta.go @@ -0,0 +1,64 @@ +// SPDX-License-Identifier: Apache-2.0 + +package mcp + +import ( + "context" + "encoding/json" + "fmt" + + "github.com/complytime/complypack/internal/requirement" + "github.com/modelcontextprotocol/go-sdk/mcp" +) + +func createAnalyzeParameterDeltaTool() *mcp.Tool { + return &mcp.Tool{ + Name: "analyze_parameter_delta", + Description: "Gather parameter comparisons across a resolved policy. Returns structured L3 parameter values alongside the L1/L2 requirement text they map to. The caller interprets the relationship — the tool does not judge.", + InputSchema: map[string]interface{}{ + "type": "object", + "properties": map[string]interface{}{ + "policyName": map[string]interface{}{ + "type": "string", + "description": "Name of the resolved policy to analyze", + }, + }, + "required": []interface{}{"policyName"}, + }, + } +} + +func handleAnalyzeParameterDelta(store *ResourceStore, artifactSet *requirement.ArtifactSet) mcp.ToolHandler { + return func(ctx context.Context, req *mcp.CallToolRequest) (*mcp.CallToolResult, error) { + var input struct { + PolicyName string `json:"policyName"` + } + + if err := json.Unmarshal(req.Params.Arguments, &input); err != nil { + return nil, fmt.Errorf("invalid input: %w", err) + } + + rp, found := store.resolved[input.PolicyName] + if !found { + return nil, fmt.Errorf("policy %q not found", input.PolicyName) + } + + report, err := requirement.AnalyzeDelta(rp, artifactSet) + if err != nil { + return nil, fmt.Errorf("delta analysis failed: %w", err) + } + + responseData, err := json.Marshal(report) + if err != nil { + return nil, fmt.Errorf("failed to marshal response: %w", err) + } + + return &mcp.CallToolResult{ + Content: []mcp.Content{ + &mcp.TextContent{ + Text: string(responseData), + }, + }, + }, nil + } +} diff --git a/internal/mcp/tool_delta_test.go b/internal/mcp/tool_delta_test.go new file mode 100644 index 0000000..fb205ca --- /dev/null +++ b/internal/mcp/tool_delta_test.go @@ -0,0 +1,152 @@ +// SPDX-License-Identifier: Apache-2.0 + +package mcp + +import ( + "context" + "encoding/json" + "testing" + + "github.com/complytime/complypack/internal/requirement" + "github.com/gemaraproj/go-gemara" + "github.com/modelcontextprotocol/go-sdk/mcp" + "github.com/stretchr/testify/assert" + "github.com/stretchr/testify/require" +) + +func testDeltaStore() (*ResourceStore, *requirement.ArtifactSet) { + catalog := &gemara.ControlCatalog{ + Metadata: gemara.Metadata{Id: "container-baseline"}, + Controls: []gemara.Control{ + { + Id: "CTL-TLS-001", + Title: "TLS Configuration", + AssessmentRequirements: []gemara.AssessmentRequirement{ + {Id: "CTL-TLS-001-AR1", Text: "TLS minimum version"}, + }, + }, + }, + } + + policy := &gemara.Policy{ + Metadata: gemara.Metadata{ + Id: "org-policy", + MappingReferences: []gemara.MappingReference{ + {Id: "container-baseline"}, + }, + }, + Imports: gemara.Imports{ + Catalogs: []gemara.CatalogImport{ + {ReferenceId: "container-baseline"}, + }, + }, + Adherence: gemara.Adherence{ + AssessmentPlans: []gemara.AssessmentPlan{ + { + RequirementId: "CTL-TLS-001-AR1", + Parameters: []gemara.Parameter{ + {Label: "tls_minimum_version", AcceptedValues: []string{"1.3"}}, + }, + }, + }, + }, + } + + set := &requirement.ArtifactSet{ + Catalogs: map[string]*gemara.ControlCatalog{"container-baseline": catalog}, + Policies: map[string]*gemara.Policy{"org-policy": policy}, + Guidance: make(map[string]*gemara.GuidanceCatalog), + Mappings: make(map[string]*gemara.MappingDocument), + } + + rp, _ := requirement.ResolvePolicy(*policy, set) + + store := &ResourceStore{ + artifacts: map[string]any{"container-baseline": catalog, "org-policy": policy}, + resolved: map[string]*requirement.ResolvedPolicy{"org-policy": rp}, + schemas: map[string][]byte{}, + } + + return store, set +} + +func TestHandleAnalyzeParameterDelta(t *testing.T) { + store, set := testDeltaStore() + handler := handleAnalyzeParameterDelta(store, set) + + t.Run("successful analysis", func(t *testing.T) { + input := map[string]interface{}{"policyName": "org-policy"} + inputJSON, err := json.Marshal(input) + require.NoError(t, err) + + req := &mcp.CallToolRequest{ + Params: &mcp.CallToolParamsRaw{ + Arguments: json.RawMessage(inputJSON), + }, + } + + result, err := handler(context.Background(), req) + require.NoError(t, err) + require.NotNil(t, result) + require.Len(t, result.Content, 1) + + textContent, ok := result.Content[0].(*mcp.TextContent) + require.True(t, ok) + + var response map[string]interface{} + err = json.Unmarshal([]byte(textContent.Text), &response) + require.NoError(t, err) + + assert.Equal(t, "org-policy", response["policy"]) + comparisons, ok := response["comparisons"].([]interface{}) + require.True(t, ok) + assert.Len(t, comparisons, 1) + + first := comparisons[0].(map[string]interface{}) + assert.Equal(t, "CTL-TLS-001-AR1", first["requirement_id"]) + assert.Equal(t, "1.3", first["policy_value"]) + assert.Equal(t, "TLS minimum version", first["requirement_text"]) + }) + + t.Run("policy not found", func(t *testing.T) { + input := map[string]interface{}{"policyName": "nonexistent"} + inputJSON, _ := json.Marshal(input) + req := &mcp.CallToolRequest{ + Params: &mcp.CallToolParamsRaw{ + Arguments: json.RawMessage(inputJSON), + }, + } + + result, err := handler(context.Background(), req) + assert.Error(t, err) + assert.Nil(t, result) + assert.Contains(t, err.Error(), "not found") + }) + + t.Run("invalid input", func(t *testing.T) { + req := &mcp.CallToolRequest{ + Params: &mcp.CallToolParamsRaw{ + Arguments: json.RawMessage([]byte(`{invalid`)), + }, + } + + result, err := handler(context.Background(), req) + assert.Error(t, err) + assert.Nil(t, result) + }) +} + +func TestCreateAnalyzeParameterDeltaTool(t *testing.T) { + tool := createAnalyzeParameterDeltaTool() + assert.Equal(t, "analyze_parameter_delta", tool.Name) + assert.NotEmpty(t, tool.Description) + + schema, ok := tool.InputSchema.(map[string]interface{}) + require.True(t, ok) + assert.Equal(t, "object", schema["type"]) + + props, ok := schema["properties"].(map[string]interface{}) + require.True(t, ok) + _, ok = props["policyName"] + assert.True(t, ok) +} diff --git a/internal/requirement/classify.go b/internal/requirement/classify.go index 9088a53..df34335 100644 --- a/internal/requirement/classify.go +++ b/internal/requirement/classify.go @@ -14,6 +14,7 @@ type ArtifactSet struct { Catalogs map[string]*gemara.ControlCatalog Policies map[string]*gemara.Policy Guidance map[string]*gemara.GuidanceCatalog + Mappings map[string]*gemara.MappingDocument } // NewArtifactSet returns an initialized ArtifactSet. @@ -22,6 +23,7 @@ func NewArtifactSet() *ArtifactSet { Catalogs: make(map[string]*gemara.ControlCatalog), Policies: make(map[string]*gemara.Policy), Guidance: make(map[string]*gemara.GuidanceCatalog), + Mappings: make(map[string]*gemara.MappingDocument), } } @@ -52,6 +54,12 @@ func Classify(data ...[]byte) (*ArtifactSet, error) { return nil, fmt.Errorf("artifact %d (GuidanceCatalog): %w", i, err) } as.Guidance[gc.Metadata.Id] = &gc + case gemara.MappingDocumentArtifact: + var md gemara.MappingDocument + if err := goyaml.Unmarshal(d, &md); err != nil { + return nil, fmt.Errorf("artifact %d (MappingDocument): %w", i, err) + } + as.Mappings[md.Metadata.Id] = &md } } return as, nil @@ -78,5 +86,11 @@ func (as *ArtifactSet) Merge(other *ArtifactSet) error { } as.Guidance[id] = gc } + for id, md := range other.Mappings { + if _, exists := as.Mappings[id]; exists { + return fmt.Errorf("duplicate artifact id %q across sources", id) + } + as.Mappings[id] = md + } return nil } diff --git a/internal/requirement/delta.go b/internal/requirement/delta.go new file mode 100644 index 0000000..2ddb1f9 --- /dev/null +++ b/internal/requirement/delta.go @@ -0,0 +1,105 @@ +// SPDX-License-Identifier: Apache-2.0 + +package requirement + +import "fmt" + +// ParameterComparison pairs a structured L3 parameter with the +// L1/L2 requirement text it maps to. The caller interprets the +// relationship — the engine does not judge. +type ParameterComparison struct { + // RequirementID is the assessment requirement this comparison targets (e.g. "CTL-TLS-001-AR1"). + RequirementID string `json:"requirement_id"` + // Label is the parameter name from the policy's assessment plan (e.g. "tls_minimum_version"). + Label string `json:"label"` + // PolicyValue is the concrete value the L3 policy sets for this parameter. + PolicyValue string `json:"policy_value"` + // PolicySource is the ID of the policy that provides the parameter value. + PolicySource string `json:"policy_source"` + // RequirementText is the L1/L2 assessment requirement text from the catalog. + RequirementText string `json:"requirement_text"` + // CatalogSource is the ID of the catalog that defines the requirement. + CatalogSource string `json:"catalog_source"` +} + +// DeltaReport is the result of gathering parameter comparisons +// across a resolved policy. +type DeltaReport struct { + PolicyID string `json:"policy"` + CatalogsCompared []string `json:"catalogs_compared"` + Comparisons []ParameterComparison `json:"comparisons"` +} + +// AnalyzeDelta gathers L3 parameter values alongside the L1/L2 +// requirement text they map to. Returns structured pairs for the +// caller to interpret — no verdicts or heuristics. +func AnalyzeDelta(rp *ResolvedPolicy, set *ArtifactSet) (*DeltaReport, error) { + if rp == nil { + return nil, fmt.Errorf("resolved policy is nil") + } + + var catalogIDs []string + for _, cat := range rp.ControlCatalogs { + catalogIDs = append(catalogIDs, cat.Metadata.Id) + } + + reqTextIndex := buildRequirementTextIndex(rp) + + var comparisons []ParameterComparison + for _, plan := range rp.Policy.Adherence.AssessmentPlans { + for _, param := range plan.Parameters { + policyValue := "" + if len(param.AcceptedValues) > 0 { + policyValue = param.AcceptedValues[0] + } + + reqText, catalogSource := reqTextIndex.lookup(plan.RequirementId) + + comparisons = append(comparisons, ParameterComparison{ + RequirementID: plan.RequirementId, + Label: param.Label, + PolicyValue: policyValue, + PolicySource: rp.Policy.Metadata.Id, + RequirementText: reqText, + CatalogSource: catalogSource, + }) + } + } + + return &DeltaReport{ + PolicyID: rp.Policy.Metadata.Id, + CatalogsCompared: catalogIDs, + Comparisons: comparisons, + }, nil +} + +type requirementTextIndex struct { + texts map[string]string + sources map[string]string +} + +func buildRequirementTextIndex(rp *ResolvedPolicy) requirementTextIndex { + idx := requirementTextIndex{ + texts: make(map[string]string), + sources: make(map[string]string), + } + for _, cat := range rp.ControlCatalogs { + for _, ctrl := range cat.Controls { + for _, ar := range ctrl.AssessmentRequirements { + idx.texts[ar.Id] = ar.Text + idx.sources[ar.Id] = cat.Metadata.Id + } + } + } + for _, gc := range rp.GuidanceCatalogs { + for _, gl := range gc.Guidelines { + idx.texts[gl.Id] = gl.Objective + idx.sources[gl.Id] = gc.Metadata.Id + } + } + return idx +} + +func (idx requirementTextIndex) lookup(reqID string) (text, source string) { + return idx.texts[reqID], idx.sources[reqID] +} diff --git a/internal/requirement/delta_test.go b/internal/requirement/delta_test.go new file mode 100644 index 0000000..1004a36 --- /dev/null +++ b/internal/requirement/delta_test.go @@ -0,0 +1,104 @@ +// SPDX-License-Identifier: Apache-2.0 + +package requirement + +import ( + "testing" + + "github.com/gemaraproj/go-gemara" + "github.com/stretchr/testify/assert" + "github.com/stretchr/testify/require" +) + +func testDeltaArtifactSet() *ArtifactSet { + catalog := &gemara.ControlCatalog{ + Metadata: gemara.Metadata{Id: "container-baseline"}, + Controls: []gemara.Control{ + { + Id: "CTL-TLS-001", + Title: "TLS Configuration", + AssessmentRequirements: []gemara.AssessmentRequirement{ + {Id: "CTL-TLS-001-AR1", Text: "TLS minimum version must be enforced"}, + }, + }, + { + Id: "CTL-CERT-001", + Title: "Certificate Management", + AssessmentRequirements: []gemara.AssessmentRequirement{ + {Id: "CTL-CERT-001-AR1", Text: "Certificate validity must not exceed maximum"}, + }, + }, + }, + } + + policy := &gemara.Policy{ + Metadata: gemara.Metadata{ + Id: "org-parent-policy", + MappingReferences: []gemara.MappingReference{ + {Id: "container-baseline"}, + }, + }, + Imports: gemara.Imports{ + Catalogs: []gemara.CatalogImport{ + {ReferenceId: "container-baseline"}, + }, + }, + Adherence: gemara.Adherence{ + AssessmentPlans: []gemara.AssessmentPlan{ + { + RequirementId: "CTL-TLS-001-AR1", + Parameters: []gemara.Parameter{ + {Label: "tls_minimum_version", AcceptedValues: []string{"1.3"}}, + }, + }, + { + RequirementId: "CTL-CERT-001-AR1", + Parameters: []gemara.Parameter{ + {Label: "max_validity_days", AcceptedValues: []string{"90"}}, + }, + }, + }, + }, + } + + return &ArtifactSet{ + Catalogs: map[string]*gemara.ControlCatalog{"container-baseline": catalog}, + Policies: map[string]*gemara.Policy{"org-parent-policy": policy}, + Guidance: make(map[string]*gemara.GuidanceCatalog), + Mappings: make(map[string]*gemara.MappingDocument), + } +} + +func TestAnalyzeDelta(t *testing.T) { + set := testDeltaArtifactSet() + policy := set.Policies["org-parent-policy"] + + rp, err := ResolvePolicy(*policy, set) + require.NoError(t, err) + + report, err := AnalyzeDelta(rp, set) + require.NoError(t, err) + + assert.Equal(t, "org-parent-policy", report.PolicyID) + assert.Contains(t, report.CatalogsCompared, "container-baseline") + assert.Len(t, report.Comparisons, 2) + + tls := report.Comparisons[0] + assert.Equal(t, "CTL-TLS-001-AR1", tls.RequirementID) + assert.Equal(t, "tls_minimum_version", tls.Label) + assert.Equal(t, "1.3", tls.PolicyValue) + assert.Equal(t, "org-parent-policy", tls.PolicySource) + assert.Equal(t, "TLS minimum version must be enforced", tls.RequirementText) + assert.Equal(t, "container-baseline", tls.CatalogSource) + + cert := report.Comparisons[1] + assert.Equal(t, "CTL-CERT-001-AR1", cert.RequirementID) + assert.Equal(t, "90", cert.PolicyValue) + assert.Equal(t, "Certificate validity must not exceed maximum", cert.RequirementText) +} + +func TestAnalyzeDelta_NilPolicy(t *testing.T) { + set := testDeltaArtifactSet() + _, err := AnalyzeDelta(nil, set) + assert.Error(t, err) +} diff --git a/internal/requirement/resolve_test.go b/internal/requirement/resolve_test.go index 43fc82f..1dfa37d 100644 --- a/internal/requirement/resolve_test.go +++ b/internal/requirement/resolve_test.go @@ -59,6 +59,7 @@ func testArtifactSet() *ArtifactSet { Catalogs: map[string]*gemara.ControlCatalog{"test-catalog": catalog}, Policies: map[string]*gemara.Policy{"test-policy": policy}, Guidance: make(map[string]*gemara.GuidanceCatalog), + Mappings: make(map[string]*gemara.MappingDocument), } } diff --git a/internal/requirement/resolved_policy.go b/internal/requirement/resolved_policy.go index 9a4a846..33a23c2 100644 --- a/internal/requirement/resolved_policy.go +++ b/internal/requirement/resolved_policy.go @@ -86,3 +86,15 @@ func (rp *ResolvedPolicy) ControlIDs() []string { func (rp *ResolvedPolicy) ParametersForRequirement(reqID string) []gemara.Parameter { return rp.paramIndex[reqID] } + +// ImportedGuidanceIDs returns the metadata IDs of guidance catalogs +// imported by this policy. Guidance catalogs loaded in the artifact set +// but not in this list are "under evaluation" — available for crosswalk +// but not mandated. +func (rp *ResolvedPolicy) ImportedGuidanceIDs() []string { + ids := make([]string, 0, len(rp.GuidanceCatalogs)) + for _, gc := range rp.GuidanceCatalogs { + ids = append(ids, gc.Metadata.Id) + } + return ids +} diff --git a/internal/requirement/resolved_policy_test.go b/internal/requirement/resolved_policy_test.go index 57fc835..e8d05fa 100644 --- a/internal/requirement/resolved_policy_test.go +++ b/internal/requirement/resolved_policy_test.go @@ -5,6 +5,7 @@ package requirement import ( "testing" + "github.com/gemaraproj/go-gemara" "github.com/stretchr/testify/assert" "github.com/stretchr/testify/require" ) @@ -60,3 +61,56 @@ func TestResolvedPolicy_QueryMethods(t *testing.T) { assert.Empty(t, params) }) } + +func TestResolvedPolicy_ImportedGuidanceIDs(t *testing.T) { + guidanceCatalog := &gemara.GuidanceCatalog{ + Metadata: gemara.Metadata{Id: "guidance-1"}, + Guidelines: []gemara.Guideline{ + {Id: "GL-001", Title: "Test guideline"}, + }, + } + + policy := &gemara.Policy{ + Metadata: gemara.Metadata{ + Id: "test-policy", + MappingReferences: []gemara.MappingReference{ + {Id: "test-catalog"}, + {Id: "guidance-1"}, + }, + }, + Imports: gemara.Imports{ + Catalogs: []gemara.CatalogImport{ + {ReferenceId: "test-catalog"}, + }, + Guidance: []gemara.GuidanceImport{ + {ReferenceId: "guidance-1"}, + }, + }, + Adherence: gemara.Adherence{}, + } + + catalog := &gemara.ControlCatalog{ + Metadata: gemara.Metadata{Id: "test-catalog"}, + Controls: []gemara.Control{ + { + Id: "CTRL-001", + AssessmentRequirements: []gemara.AssessmentRequirement{ + {Id: "REQ-001", Text: "Verify"}, + }, + }, + }, + } + + set := &ArtifactSet{ + Catalogs: map[string]*gemara.ControlCatalog{"test-catalog": catalog}, + Policies: map[string]*gemara.Policy{"test-policy": policy}, + Guidance: map[string]*gemara.GuidanceCatalog{"guidance-1": guidanceCatalog}, + Mappings: make(map[string]*gemara.MappingDocument), + } + + rp, err := ResolvePolicy(*policy, set) + require.NoError(t, err) + + ids := rp.ImportedGuidanceIDs() + assert.Equal(t, []string{"guidance-1"}, ids) +} diff --git a/skills/complypack/SKILL.md b/skills/complypack/SKILL.md deleted file mode 100644 index 9f8bed8..0000000 --- a/skills/complypack/SKILL.md +++ /dev/null @@ -1,366 +0,0 @@ ---- -name: complypack -description: Use when user mentions complypack, wants to generate Rego policies from Gemara catalogs, extract assessment requirements and parameters, or work with compliance validation for Kubernetes, Terraform, Docker, Ansible, or CI platforms ---- - -# ComplyPack: Gemara Policy Generation and Assessment - -## Overview - -Generate Rego policies from Gemara control catalogs that enforce compliance requirements. Policies must be written to disk, validated against the target platform schema, and tested with sample inputs. - -**Core principle:** Read control definitions from source → Generate platform-specific policy → Write to disk → Verify it works. - -## When to Use - -Use when: -- User requests "generate policy for control X" -- User specifies a Gemara catalog and target platform -- User mentions Conftest, OPA, or Rego -- Generating compliance policies from security frameworks - -Do NOT use for: -- Writing arbitrary Rego policies (not from Gemara controls) -- Generating policies without a source catalog -- One-off policy snippets that don't need disk storage - -## Quick Reference - -| Step | Action | Output | -|------|--------|--------| -| 1. Read control | Get definition from catalog (MCP/file/API) | Control text, ID, title | -| 2. Get parameters | Extract assessment requirements with test parameters | Thresholds, values, tools | -| 3. Read schema | Get platform schema (MCP/file) | JSON Schema or CUE | -| 4. Choose format | OPA (allow) or Conftest (deny) | Policy structure | -| 5. Generate policy | Write Rego with control mapping and parameters | .rego file | -| 6. Write to disk | Save to `policy/` or user-specified path | File on disk | -| 7. Verify | Test with sample input | Pass/fail results | - -## The Process - -### Step 1: Read Control Definition from Source - -**DO NOT generate from general knowledge.** Always read the actual control text. - -**If ComplyPack MCP server available:** -``` -1. List available catalogs: ListMcpResourcesTool(server="complypack") -2. Read specific control: ReadMcpResourceTool(server="complypack", uri="complypack://catalog/{name}") -3. Extract control ID, title, description -``` - -**If catalog is a file:** -``` -1. Read catalog YAML/JSON -2. Find control by ID -3. Extract control text -``` - -**Critical:** The control definition is your requirements specification. Don't improvise. - -### Step 2: Get Assessment Requirements and Parameters - -**IMPORTANT:** Assessment requirements contain test parameters, thresholds, and approved tools. - -**If ComplyPack MCP server available AND catalog is a Policy:** -``` -Use get_assessment_requirements tool: -{ - "catalogName": "policy-name", - "controlId": "CTL-XXX-001" // optional filter -} - -Returns: -- Structured parameters from Policy.Adherence.AssessmentPlans -- Parameter labels, descriptions, accepted values -- Tool mentions (ToolA, ToolB, etc.) -- File patterns (.gitlab-ci.yml, Jenkinsfile) -``` - -**What you get:** -- `Parameters` - Structured test values from assessment plans (e.g., {"timeout": "60"}) -- `Tools` - Approved tools mentioned (ToolA, ToolB, ToolC) -- `TestValues` - Algorithms, config files, permissions patterns -- `Text` - Full requirement text for context - -**Example:** -```json -{ - "id": "CTL-DATA-001-AR3", - "control_id": "CTL-DATA-001", - "text": "Certificate validity must not exceed maximum", - "parameters": { - "max_validity_days": "90", - "max_validity_days_description": "Maximum certificate lifetime" - }, - "tools": [], - "test_values": [] -} -``` - -**Use these parameters in your policy:** -- Thresholds → Use exact values in comparisons -- Tools → Reference in validation logic -- Accepted values → Use in allow/deny rules - -**If catalog is a ControlCatalog (not Policy):** -- Assessment requirements exist but parameters are not attached -- You'll get requirement text and hints (tools, patterns) -- No structured parameter values - -### Step 3: Read Platform Schema - -Understand what data structure the policy will evaluate. - -**If ComplyPack MCP server available:** -``` -ReadMcpResourceTool(server="complypack", uri="complypack://schema/{platform}") -``` - -**Platforms:** kubernetes, terraform, docker, ansible, ci - -**Schema tells you:** -- Available fields to validate -- Data types and structure -- What's actually in the input - -### Step 4: Choose Policy Format - -**Ask user if unclear**, otherwise use this decision tree: - -```dot -digraph format_choice { - "User specified format?" [shape=diamond]; - "Mentions Conftest?" [shape=diamond]; - "Use Conftest format (deny)" [shape=box]; - "Use OPA format (allow)" [shape=box]; - - "User specified format?" -> "Use specified format" [label="yes"]; - "User specified format?" -> "Mentions Conftest?" [label="no"]; - "Mentions Conftest?" -> "Use Conftest format (deny)" [label="yes"]; - "Mentions Conftest?" -> "Use OPA format (allow)" [label="no"]; -} -``` - -**Conftest format:** -```rego -package main - -deny[msg] { - # Violation condition - msg := "Violation message" -} -``` - -**OPA format:** -```rego -package platform.controlid - -default allow := false - -allow { - # Compliance conditions -} - -violations[msg] { - # Generate violation messages -} -``` - -### Step 5: Generate the Policy - -Map control requirements to platform-specific checks: - -**Template structure:** -```rego -# {Control-ID}: {Control Title} -# {Control Description} - -package {namespace} - -import rego.v1 - -# METADATA -# custom: -# control_id: {ID} -# control_title: {Title} -# severity: {high|medium|low} - -# [Policy logic based on control requirements and platform schema] -``` - -**Key requirements:** -- Reference control ID and title in comments -- Use fields that exist in platform schema -- Write clear violation messages -- Include both positive checks (what must exist) and negative checks (what must not) - -### Step 6: Write to Disk - -**DO NOT just output to chat.** Policies must be saved to files. - -**Default structure for Conftest:** -``` -policy/ - {control-id}.rego # e.g., ac-1.rego - {control-id}_test.rego # Optional: unit tests -``` - -**Default structure for OPA:** -``` -policies/ - {platform}/ - {control-id}.rego # e.g., kubernetes/ac-1.rego -``` - -**Ask user for path if:** -- They have existing policy directory -- Project structure is unclear -- Multiple controls being generated - -### Step 7: Verify Policy Works - -**Critical step - don't skip this.** - -Create sample input matching platform schema: - -**For Kubernetes:** -```yaml -apiVersion: apps/v1 -kind: Deployment -metadata: - name: test-app -spec: - # ... based on schema -``` - -**For Terraform:** -```json -{ - "address": "aws_s3_bucket.example", - "type": "aws_s3_bucket", - "values": { - # ... based on schema - } -} -``` - -**Test the policy:** -```bash -# Conftest -conftest test input.yaml -p policy/ - -# OPA -opa eval --data policy.rego --input input.json "data.{package}.allow" -``` - -**Report results:** -- ✅ Policy syntax valid (opa check) -- ✅ Compliant input passes -- ✅ Non-compliant input fails with clear message - -## Common Mistakes - -| Mistake | Fix | -|---------|-----| -| Generated from general knowledge | Always read actual control definition from source | -| Policy only in chat | Write to disk with proper filename | -| No verification | Test with sample input before claiming done | -| Wrong format (allow vs deny) | Ask user or check for "Conftest" mention | -| Ignoring platform schema | Read schema, use actual fields that exist | -| Missing violation messages | Every deny/violation needs clear message | -| Overly complex structure | Start with single .rego file per control | - -## Red Flags - Check These Before Claiming Done - -- [ ] Did you read the control definition from the actual source? -- [ ] Did you read the platform schema to know available fields? -- [ ] Did you write the policy to disk (not just chat)? -- [ ] Did you test it with sample input? -- [ ] Does it have clear violation messages? -- [ ] Is the format correct (Conftest deny vs OPA allow)? - -## Example Workflow - -User: "Generate policy for CTL-DATA-001-AR3 from my-security-policy targeting Kubernetes" - -**Steps:** -1. ✅ Read control: `ReadMcpResourceTool(server="complypack", uri="complypack://catalog/my-security-policy")` -2. ✅ Extract CTL-DATA-001 requirements text -3. ✅ Get parameters: `get_assessment_requirements({catalogName: "my-security-policy", controlId: "CTL-DATA-001"})` -4. ✅ Extract parameter: `{"max_validity_days": "90"}` from assessment plan -5. ✅ Read schema: `ReadMcpResourceTool(server="complypack", uri="complypack://schema/kubernetes")` -6. ✅ Note schema fields: spec.tls.secretName, spec.tls.hosts -7. ✅ Generate OPA policy using `max_validity_days` parameter -8. ✅ Write to `policy/ctl-data-001-ar3.rego` -9. ✅ Create test input (Ingress with cert) -10. ✅ Run: `conftest test test-ingress.yaml -p policy/` -11. ✅ Report: "Policy enforces 90-day cert validity. Tested against sample Ingress." - -**Example policy using parameters:** -```rego -package kubernetes.ctl_data_001_ar3 - -import rego.v1 - -# CTL-DATA-001-AR3: Certificate validity -# Parameters from assessment plan: max_validity_days = 90 - -deny[msg] { - cert := input.spec.tls[_] - cert_days := get_cert_validity_days(cert.secretName) - - # Use parameter from assessment plan - max_days := 90 - cert_days > max_days - - msg := sprintf("Certificate %s exceeds maximum validity of %d days", - [cert.secretName, max_days]) -} -``` - -**NOT:** -1. ❌ Generate policy from general knowledge -2. ❌ Hardcode generic value when parameter is specified -3. ❌ Skip get_assessment_requirements step -4. ❌ Output policy in chat only -5. ❌ Skip testing - -## Multi-Control Generation - -When generating multiple controls: - -**Option 1: Separate files (recommended)** -``` -policy/ - ac-1.rego - ac-2.rego - sc-1.rego -``` -Easier to maintain, test, and selectively apply. - -**Option 2: Combined file** -``` -policy/ - all-controls.rego # Multiple deny[] or multiple packages -``` -Only if user explicitly requests combined. - -**Always:** -- Process controls one at a time -- Write each to disk before moving to next -- Test each independently - -## Tool Integration - -**Works with any tool that can:** -- Read structured data (JSON/YAML) -- Generate Rego code -- Write files to disk - -**Not specific to:** -- Claude MCP server (catalog could be files, API, etc.) -- Conftest vs OPA (support both) -- Specific platforms (works for any with schema) - -**The workflow is platform-agnostic** - adapt to your environment. diff --git a/skills/complypack/mcp.jsonc b/skills/complypack/mcp.jsonc deleted file mode 100644 index 6b7c259..0000000 --- a/skills/complypack/mcp.jsonc +++ /dev/null @@ -1,8 +0,0 @@ -{ - "mcpServers": { - "complypack": { - "command": "complypack", - "args": ["mcp", "serve", "--config", "complypack.yaml"] - } - } -} diff --git a/skills/pack/SKILL.md b/skills/pack/SKILL.md new file mode 100644 index 0000000..bf8fa52 --- /dev/null +++ b/skills/pack/SKILL.md @@ -0,0 +1,46 @@ +--- +name: pack +description: Use when user wants to generate Rego policies from Gemara catalogs, extract assessment requirements and parameters, or work with compliance validation for Kubernetes, Terraform, Docker, Ansible, or CI platforms +--- + +# /comply:pack — Rego Policy Generation and Assessment + +Generate Rego policies from Gemara Control Catalogs that enforce compliance requirements. Policies must be written to disk, validated against the target platform schema, and tested with sample inputs. + +**Core principle:** Read control definitions from source → Generate platform-specific policy → Write to disk → Verify it works. + +## When to Use + +- User requests "generate policy for control X" +- User specifies a Gemara catalog and target platform +- User mentions Conftest, OPA, or Rego +- Generating compliance policies from security frameworks + +Do NOT use for: +- Writing arbitrary Rego policies (not from Gemara controls) +- Generating policies without a source catalog + +## Quick Reference + +| Step | Action | Output | +| ---- | ------ | ------ | +| 1. Read control | Get definition from catalog (MCP) | Control text, ID, title | +| 2. Get parameters | Extract assessment requirements | Thresholds, values | +| 3. Read schema | Get platform schema (MCP) | JSON Schema or CUE | +| 4. Choose format | OPA (allow) or Conftest (deny) | Policy structure | +| 5. Generate policy | Write Rego with control mapping | .rego file | +| 6. Write to disk | Save to `policy/` | File on disk | +| 7. Verify | Test with sample input | Pass/fail results | + +## Safety + +**DO NOT generate from general knowledge.** Always read the actual control text from MCP. + +## MCP Resources and Tools + +- `complypack://catalog/*` — Control Catalogs, Guidance Catalogs, Policies +- `complypack://schema/*` — Platform schemas +- `complypack://evaluator` — Available evaluators +- `get_assessment_requirements` — Extract assessment requirements with parameters +- `validate_policy` — Validate policy syntax and contract compliance +- `test_policy` — Run policy tests against sample data diff --git a/skills/pipeline/SKILL.md b/skills/pipeline/SKILL.md new file mode 100644 index 0000000..ee43bcd --- /dev/null +++ b/skills/pipeline/SKILL.md @@ -0,0 +1,52 @@ +--- +name: pipeline +description: Use when user wants to build Gemara Policy artifacts for audit preparation or compliance program setup +--- + +# /comply:pipeline — ComplyTime Audit Pipeline + +Guide users through building a Gemara Policy (applicability statement) from their system architecture and governance sources. The Gemara Policy is the formal contract between audit and engineering, functionally equivalent to an ISO 27001 Statement of Applicability or a NIST System Security Plan. + +## Safety + +**CRITICAL:** Every stage MUST read control IDs, requirement IDs, and parameter values from MCP resources. DO NOT generate these from memory. The MCP server is the source of truth. + +## Pipeline Stages + +| Stage | Artifact | Purpose | +|-----------|-----------------------------------|------------------------------------------------------------------| +| scoping | `.complytime/scoping.yaml` | System profile + Control Catalog scoping + gap analysis | +| mapping | `.complytime/delta-report.yaml` | Parameter delta analysis + harmonization across framework layers | +| adherence | `.complytime/child-policy.yaml` | Compile the child Policy with adherence plan | + +After adherence, invoke `/comply:pack` to generate assessment logic for use with `complyctl`. + +## Router Logic + +1. Check if `.complytime/` directory exists and which artifacts are present +2. Determine pipeline state: + - No `.complytime/` directory → start at **scoping** + - `scoping.yaml` exists but no `delta-report.yaml` → offer **mapping** + - `delta-report.yaml` exists but no `child-policy.yaml` → offer **adherence** + - `child-policy.yaml` exists → pipeline complete, offer to re-run any stage or proceed to `/comply:pack` +3. If the user specified a stage, validate prerequisites: + - **mapping** requires `scoping.yaml` + - **adherence** requires `delta-report.yaml` +4. Dispatch to the appropriate stage skill + +## Dispatching + +Read the stage instructions from this skill's base directory before proceeding: + +- **scoping** → `scoping.md` +- **mapping** → `mapping.md` +- **adherence** → `adherence.md` + +## Status Display + +```text +/comply:pipeline status: + [done] scoping — .complytime/scoping.yaml + [done] mapping — .complytime/delta-report.yaml + [next] adherence — not yet run +``` diff --git a/skills/pipeline/adherence.md b/skills/pipeline/adherence.md new file mode 100644 index 0000000..2275487 --- /dev/null +++ b/skills/pipeline/adherence.md @@ -0,0 +1,123 @@ +--- +name: comply-adherence +description: Populate a Gemara Policy defining what controls apply, with what parameter values, and how evidence will be collected +user-invocable: false +--- + +# Adherence — Compile Policy + +Compile or alter a Gemara Policy artifact. Declares what controls apply, with exact parameter values, and defines the adherence plan: frequency, evaluation method, and evidence requirements. + +Evidence collection and Evaluation Logs are produced by `complyctl` at runtime. + +## Prerequisites + +- `.complytime/scoping.yaml` +- `.complytime/delta-report.yaml` + +Verify all parameters are resolved (no `pending_user_decision`). If unresolved, tell the user to re-run mapping. + +## Process + +### Step 1: Read Input Artifacts + +Read both `.complytime/scoping.yaml` and `.complytime/delta-report.yaml`. + +### Step 2: Build Mapping References + +From the delta report's `sources`, build `mapping_references`: +- One for the parent Policy +- One for each scoped Control Catalog +- One for each Guidance Catalog for the target framework + +### Step 3: Build Imports + +- `imports.catalogs` — one per Control Catalog +- `imports.guidance` — one per mandated Guidance Catalog + +### Step 4: Build Assessment Plans + +Group by `requirement-id`. Each plan: +- `id` — unique plan identifier +- `requirement-id` — the assessment requirement this plan addresses +- `frequency` (e.g., "30d", "90d", "365d") +- `evaluation-methods` — list of `{id, type: Behavioral|Intent, mode: Automated|Manual}` +- `evidence-requirements` — what evidence is collected +- `parameters` — frozen values from harmonization, each with `id`, `label`, `accepted-values`, `description` + +### Step 5: Compile the Policy + +```yaml +title: " Policy" +metadata: + id: -policy + gemara-version: v1.0.0 + type: Policy + description: "" + author: + id: + name: + type: Software Assisted + mapping-references: + - id: + title: "" + version: "" +contacts: + responsible: + - name: "" + accountable: + - name: "" +scope: + in: + technologies: + - + groups: + - +imports: + catalogs: + - reference-id: + guidance: + - reference-id: +adherence: + assessment-plans: + - id: + requirement-id: "" + frequency: "" + evaluation-methods: + - id: + type: Behavioral + mode: Automated + evidence-requirements: "" + parameters: + - id: + label: "