diff --git a/.github/workflows/keyfactor-starter-workflow.yml b/.github/workflows/keyfactor-starter-workflow.yml index a4649f2..0f3d3ae 100644 --- a/.github/workflows/keyfactor-starter-workflow.yml +++ b/.github/workflows/keyfactor-starter-workflow.yml @@ -11,10 +11,17 @@ on: jobs: call-starter-workflow: - uses: keyfactor/actions/.github/workflows/starter.yml@3.1.2 + uses: keyfactor/actions/.github/workflows/starter.yml@v5 + with: + command_token_url: ${{ vars.COMMAND_TOKEN_URL }} + command_hostname: ${{ vars.COMMAND_HOSTNAME }} + command_base_api_path: ${{ vars.COMMAND_API_PATH }} secrets: token: ${{ secrets.V2BUILDTOKEN}} - APPROVE_README_PUSH: ${{ secrets.APPROVE_README_PUSH}} gpg_key: ${{ secrets.KF_GPG_PRIVATE_KEY }} gpg_pass: ${{ secrets.KF_GPG_PASSPHRASE }} scan_token: ${{ secrets.SAST_TOKEN }} + entra_username: ${{ secrets.DOCTOOL_ENTRA_USERNAME }} + entra_password: ${{ secrets.DOCTOOL_ENTRA_PASSWD }} + command_client_id: ${{ secrets.COMMAND_CLIENT_ID }} + command_client_secret: ${{ secrets.COMMAND_CLIENT_SECRET }} diff --git a/.gitignore b/.gitignore index dfcfd56..f27f50f 100644 --- a/.gitignore +++ b/.gitignore @@ -3,6 +3,11 @@ ## ## Get latest from https://github.com/github/gitignore/blob/master/VisualStudio.gitignore +# Local credentials and per-machine IDE state - never commit +.secrets/ +*.env +.claude/ + # User-specific files *.rsuser *.suo diff --git a/CHANGELOG.md b/CHANGELOG.md index f760407..103f041 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1,6 +1,164 @@ +v1.2.0 +- Added Discovery job (`CertStores.GcpCertMgr.Discovery`) that enumerates all + GCP projects accessible to the orchestrator's service account and emits one + candidate store path per (project, location) pair in canonical + `projects/{projectId}/locations/{location}` form. + - The Schedule Discovery form in Keyfactor Command does not expose a + Client Machine field for GCP - Command auto-populates it (typically the + orchestrator hostname) and the orchestrator logs it for traceability + only. The actual project set is bounded by the service account's IAM + bindings, not by anything the operator types into the discovery form. + - "Directories to search" is repurposed as a comma-separated list of GCP + locations (regions). The Keyfactor Command UI requires this field be + non-empty; the recommended value is just `global`. The orchestrator also + falls back to `global` defensively if it ever receives a blank value via + a non-UI submission path. + - Service account credentials use Application Default Credentials, matching + the recommended deployment on a GCE VM / GKE pod with workload identity, + or the `GOOGLE_APPLICATION_CREDENTIALS` environment variable on the + orchestrator host when running outside GCP. +- Added `FlowLogger` and `JobBase` infrastructure shared across all three jobs + (Inventory, Management, Discovery). FlowLogger captures step-by-step traces + with timing, and the summary is appended to `JobResult.FailureMessage` on + every job result so operators can see what happened from job history alone. +- Added typed exception unwrapping (`JobBase.DescribeException`) that surfaces + `GoogleApiException` HTTP status + ErrorResponseContent through `AggregateException` + walls instead of letting them be flattened into generic `.Message`s. +- Added PAM secret resolution support via `IPAMSecretResolver` injected into + every job constructor. The existing `ServiceAccountKey` store property is a + file path (not a secret) so PAM has no effect on it today, but the plumbing + is in place for future PAM-eligible properties. +- Bumped `Keyfactor.Orchestrators.IOrchestratorJobExtensions` 0.6.0 → 0.7.0 to + pick up `IPAMSecretResolver` (no breaking changes for existing job behavior). +- Added `Google.Apis.CloudResourceManager.v3` package to support the project + enumeration that Discovery requires. + +### Known limitations +- The discovery-job ClientMachine value is auto-populated by Keyfactor Command + (typically the orchestrator hostname) and is not operator-configurable from + the Schedule Discovery dialog for GCP. It is logged for traceability but + never used by the orchestrator for filtering. If the service account has + visibility into multiple GCP organizations, Discovery emits projects from + all of them - constrain at IAM if that is not desired. +- Discovery does not probe each (project, location) candidate to confirm the + Certificate Manager API is enabled or that any certificates exist. Operators + can leave `Create Certificate Store If Missing` checked to auto-approve every + candidate and let dead-end stores fail their first inventory; or leave it + unchecked and approve only the candidates they want to track. + +### Changed (schema unification) +- Unified the store-type schema so manually-created and Discovery-approved + stores configure the same way. **Store Path** is now the single source of + truth for which Certificate Manager instance the store targets, in canonical + form `projects/{projectId}/locations/{location}`. Inventory and Management + read the GCP resource path from this field for both flows. Full design + rationale (constraint, alternatives considered, trade-offs accepted) lives + in `docsource/gcpcertmgr.md` under "Design rationale: why Store Path is the + source of truth". +- **Client Machine** is repurposed as a display-only label. The recommended + value is the GCP Organization ID; the orchestrator does not parse a project + ID out of it. Documented in the updated store-type description. +- The **Location** custom property is deprecated. New stores leave it blank; + the value is parsed out of Store Path. The field remains in the manifest + with `Required: false` and a deprecation note so existing v1.1 stores keep + rendering correctly in Command's UI. + +### Changed (authentication) +- Authentication consolidates around Application Default Credentials (ADC). + This is the only credential mechanism that works uniformly across all four + job types - the previous `Service Account Key File Path` custom store + property was readable only by Inventory/Management because the Keyfactor + Command discovery-job UI does not surface store-type custom properties. + ADC works whether the orchestrator runs inside GCP (via workload identity + / GCE metadata server) or outside GCP (via the + `GOOGLE_APPLICATION_CREDENTIALS` environment variable on the orchestrator + host). +- The `Service Account Key File Path` (`ServiceAccountKey`) custom store + property is deprecated. New stores leave it blank. v1.1 stores that have + it populated continue to work via a deprecation-logged fallback in + `GcpCertificateManagerClient.LoadCredentials`. Removal is scheduled for + v2.0. + +### Fixed +- `integration-manifest.json` previously advertised `Create: true` under + `SupportedOperations`, but `Management.cs` has only ever handled `Add` and + `Remove` - a Create job from Keyfactor Command's UI fell through to the + default case and returned `Invalid Management Operation`. The manifest now + correctly reports `Create: false`. There is no meaningful semantic for + Create on a GCP Certificate Manager store anyway: the (project, location) + container is implicit in the GCP project, so there is no per-store + "creation" the orchestrator can usefully perform. Operators who relied on + scheduling Create jobs (and getting failures) should remove those job + definitions; everyone else is unaffected. + +### Removed (docs) +- Removed three Google Cloud Console screenshot GIFs from `docsource/` that + documented the service-account creation, API enablement, and JSON-key + download flows: `ServiceAccountSettings.gif`, `ApiAccessNeeded.gif`, + `GoogleKeyJsonDownload.gif`. Replaced with verbal step-by-step + instructions and `gcloud` commands in `docsource/content.md` so the docs + do not go stale when Google redesigns the Console UI. The Keyfactor + Command store-type dialog screenshots in `docsource/images/` are + doctool-managed and remain. + +### Added (validation) +- Pre-flight alias validation in Management/Add. The orchestrator now checks + the certificate alias against GCP Certificate Manager's resource-ID rule + (`[a-z]([-a-z0-9]*[a-z0-9])?`, max 63 chars) before doing any API work or + PFX parsing. A non-conforming alias produces a clear `[FAIL] ValidateAlias` + flow step with a suggested normalized form (e.g. `Cert1` → `cert1`), + replacing the previous behavior of failing 700ms later with a wall-of-JSON + HTTP 400 from GCP. See `JobBase.ValidateGcpCertificateId`. + +### Added (Scope as entry parameter) +- Added a new `Scope` **entry parameter** (per-certificate, not per-store) that + controls the GCP Certificate Manager `scope` value on each newly-created + certificate. Previous releases hard-coded `Scope = "DEFAULT"` and never read + scope back during Inventory, which made the orchestrator unusable for + environments that depend on cross-region internal Application Load Balancers + (`ALL_REGIONS`), Media CDN (`EDGE_CACHE`), or mTLS trust-config / + authorized-client server certs (`CLIENT_AUTH`). Those customers had to + pre-create empty placeholder certificates in GCP via Terraform with the right + scope and then attach Keyfactor to the existing shell. + - Modeled as `EntryParameters` (not `Properties`) because scope is per-cert + in GCP - a single (project, location) container can legitimately hold + certificates at different scopes. A single Keyfactor store can now hold a + mix of scopes without needing one store per scope. + - Rendered as a `MultipleChoice` dropdown in Command with the four allowed + values pre-populated, so typos cannot reach the orchestrator. + - Allowed values: `DEFAULT`, `ALL_REGIONS`, `EDGE_CACHE`, `CLIENT_AUTH`. + `JobBase.ResolveScope` validates defence-in-depth: trims, uppercases, + rejects anything not in the set with a `[FAIL] ResolveScope` flow step + before any GCP API call. + - Default is `DEFAULT`. Blank, null, or missing all resolve to `DEFAULT`, + matching the pre-v1.2.1 behavior so existing stores upgrade with no + operator action. + - Inventory now reads each cert's actual `scope` from GCP's + `certificates.list` response and writes it into the cert's + `CurrentInventoryItem.Parameters` dict. GCP elides the field when the + cert is at `DEFAULT`; the orchestrator normalizes null/blank to `DEFAULT` + so Command always sees a concrete value. On subsequent renewals / + reenrollments Keyfactor replays the inventoried value into + `JobProperties`, so the cert keeps its scope through its lifecycle + automatically. + - GCP's Scope field is **create-only and immutable**. Replace (overwrite) + paths do not change scope: the `Patch` call's `UpdateMask` is `SelfManaged`, + so GCP only updates the cert/key bytes. To change a certificate's scope, + delete the certificate and re-add it. + +### Backwards compatibility +- v1.1-shape stores (Store Path blank or `n/a`, Client Machine = Project ID, + Location custom property = region) continue to work via a deprecation-logged + fallback path in `JobBase.ResolveGcpResourcePath`. Every inventory or + management run against such a store emits a single `LogWarning` naming the + store and the migration step. The fallback is scheduled for removal in v2.0. +- Migration: edit each affected store, set Store Path to + `projects/{ClientMachine-value}/locations/{Location-value}`, optionally + change Client Machine to the GCP Organization ID, optionally clear Location. + v1.1.0 - Implemented dual build for .net6/8 - Converted README to use doctool v1.0.2 -- Initial Public Version \ No newline at end of file +- Initial Public Version For Release diff --git a/GcpCertManager/Client/GcpCertificateManagerClient.cs b/GcpCertManager/Client/GcpCertificateManagerClient.cs index b41febf..bb3a242 100644 --- a/GcpCertManager/Client/GcpCertificateManagerClient.cs +++ b/GcpCertManager/Client/GcpCertificateManagerClient.cs @@ -8,6 +8,7 @@ using System.Reflection; using Google.Apis.Auth.OAuth2; using Google.Apis.CertificateManager.v1; +using Google.Apis.CloudResourceManager.v3; using Google.Apis.Services; using Google.Apis.Iam.v1; using Google.Apis.Iam.v1.Data; @@ -26,29 +27,28 @@ public CertificateManagerService GetGoogleCredentials(string credentialFileName) { ILogger _logger = LogHandler.GetClassLogger(); - //Credentials file needs to be in the same location of the executing assembly - GoogleCredential credentials; + var credentials = LoadCredentials(credentialFileName, _logger); - if (!string.IsNullOrEmpty(credentialFileName)) + var service = new CertificateManagerService(new BaseClientService.Initializer { - _logger.LogDebug("Has credential file name"); - var strExeFilePath = Assembly.GetExecutingAssembly().Location; - var strWorkPath = Path.GetDirectoryName(strExeFilePath); - var strSettingsJsonFilePath = Path.Combine(strWorkPath ?? string.Empty, credentialFileName); + HttpClientInitializer = credentials + }); - var stream = new FileStream(strSettingsJsonFilePath, - FileMode.Open - ); + return service; + } - credentials = GoogleCredential.FromStream(stream); - } - else - { - _logger.LogDebug("No credential file name"); - credentials = GoogleCredential.GetApplicationDefaultAsync().Result; - } + public CloudResourceManagerService GetCloudResourceManager(string credentialFileName) + { + ILogger _logger = LogHandler.GetClassLogger(); - var service = new CertificateManagerService(new BaseClientService.Initializer + // CloudResourceManager.search requires the cloud-platform scope when using ADC + // from a Compute Engine / GKE service account; FromStream credentials carry + // their own scopes from the JSON key. + var credentials = LoadCredentials(credentialFileName, _logger); + if (credentials.IsCreateScopedRequired) + credentials = credentials.CreateScoped(CloudResourceManagerService.Scope.CloudPlatformReadOnly); + + var service = new CloudResourceManagerService(new BaseClientService.Initializer { HttpClientInitializer = credentials }); @@ -56,6 +56,39 @@ public CertificateManagerService GetGoogleCredentials(string credentialFileName) return service; } + private static GoogleCredential LoadCredentials(string credentialFileName, ILogger logger) + { + // Credential resolution order: + // 1. Explicit ServiceAccountKey (file in the extension dir) - DEPRECATED in v1.2 + // 2. Application Default Credentials (GOOGLE_APPLICATION_CREDENTIALS env var, + // or GCE VM / GKE pod metadata server when running inside GCP) + // + // ADC is the canonical path because the Keyfactor discovery-job UI does not + // surface the per-store ServiceAccountKey custom property, so file-based auth + // can't be configured uniformly across all four job types. + if (!string.IsNullOrEmpty(credentialFileName)) + { + logger.LogWarning( + "The ServiceAccountKey store property ('{FileName}') is deprecated as of v1.2 and will be removed in v2.0. " + + "Switch to Application Default Credentials by setting GOOGLE_APPLICATION_CREDENTIALS as a machine-level " + + "environment variable on the orchestrator host (or by running the orchestrator on a GCE VM / GKE pod with " + + "workload identity), then clear this store property.", + credentialFileName); + + var strExeFilePath = Assembly.GetExecutingAssembly().Location; + var strWorkPath = Path.GetDirectoryName(strExeFilePath); + var strSettingsJsonFilePath = Path.Combine(strWorkPath ?? string.Empty, credentialFileName); + + using (var stream = new FileStream(strSettingsJsonFilePath, FileMode.Open)) + { + return GoogleCredential.FromStream(stream); + } + } + + logger.LogDebug("Using Application Default Credentials"); + return GoogleCredential.GetApplicationDefaultAsync().Result; + } + public ServiceAccountKey CreateServiceAccountKey(string serviceAccountEmail) { GoogleCredential credential = GoogleCredential.GetApplicationDefault().CreateScoped(IamService.Scope.CloudPlatform); diff --git a/GcpCertManager/FlowLogger.cs b/GcpCertManager/FlowLogger.cs new file mode 100644 index 0000000..e67b7bf --- /dev/null +++ b/GcpCertManager/FlowLogger.cs @@ -0,0 +1,243 @@ +// Copyright 2026 Keyfactor +// +// Licensed under the Apache License, Version 2.0 (the "License"); +// you may not use this file except in compliance with the License. +// You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, software +// distributed under the License is distributed on an "AS IS" BASIS, +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +// See the License for the specific language governing permissions and +// limitations under the License. + +using System; +using System.Collections.Generic; +using System.Diagnostics; +using System.Linq; +using System.Text; +using System.Threading.Tasks; +using Microsoft.Extensions.Logging; + +namespace Keyfactor.Extensions.Orchestrator.GcpCertManager +{ + public class FlowLogger : IDisposable + { + private readonly ILogger _logger; + private readonly string _flowName; + private readonly Stopwatch _overallStopwatch; + private readonly List _steps = new List(); + private readonly Stack _branchStack = new Stack(); + + public FlowLogger(ILogger logger, string flowName) + { + _logger = logger ?? throw new ArgumentNullException(nameof(logger)); + _flowName = flowName ?? throw new ArgumentNullException(nameof(flowName)); + _overallStopwatch = Stopwatch.StartNew(); + _logger.LogTrace("[FLOW:{FlowName}] === BEGIN ===", _flowName); + } + + public void Step(string name, string detail = null) + { + var step = new FlowStep { Name = name, Detail = detail, Status = StepStatus.Success }; + _steps.Add(step); + var prefix = GetPrefix(); + if (detail != null) + _logger.LogTrace("[FLOW:{FlowName}] {Prefix}[OK] {StepName} - {Detail}", _flowName, prefix, name, detail); + else + _logger.LogTrace("[FLOW:{FlowName}] {Prefix}[OK] {StepName}", _flowName, prefix, name); + } + + public void Step(string name, Action action, string detail = null) + { + var sw = Stopwatch.StartNew(); + var step = new FlowStep { Name = name, Detail = detail }; + try + { + action(); + sw.Stop(); + step.Status = StepStatus.Success; + step.ElapsedMs = sw.ElapsedMilliseconds; + _steps.Add(step); + var prefix = GetPrefix(); + _logger.LogTrace("[FLOW:{FlowName}] {Prefix}[OK] {StepName} ({Elapsed}ms){DetailSuffix}", + _flowName, prefix, name, sw.ElapsedMilliseconds, FormatDetail(detail)); + } + catch (Exception ex) + { + sw.Stop(); + step.Status = StepStatus.Failed; + step.ElapsedMs = sw.ElapsedMilliseconds; + step.ErrorMessage = ex.Message; + _steps.Add(step); + var prefix = GetPrefix(); + _logger.LogTrace("[FLOW:{FlowName}] {Prefix}[FAIL] {StepName} ({Elapsed}ms) - {Error}", + _flowName, prefix, name, sw.ElapsedMilliseconds, ex.Message); + throw; + } + } + + public async Task StepAsync(string name, Func action, string detail = null) + { + var sw = Stopwatch.StartNew(); + var step = new FlowStep { Name = name, Detail = detail }; + try + { + await action(); + sw.Stop(); + step.Status = StepStatus.Success; + step.ElapsedMs = sw.ElapsedMilliseconds; + _steps.Add(step); + var prefix = GetPrefix(); + _logger.LogTrace("[FLOW:{FlowName}] {Prefix}[OK] {StepName} ({Elapsed}ms){DetailSuffix}", + _flowName, prefix, name, sw.ElapsedMilliseconds, FormatDetail(detail)); + } + catch (Exception ex) + { + sw.Stop(); + step.Status = StepStatus.Failed; + step.ElapsedMs = sw.ElapsedMilliseconds; + step.ErrorMessage = ex.Message; + _steps.Add(step); + var prefix = GetPrefix(); + _logger.LogTrace("[FLOW:{FlowName}] {Prefix}[FAIL] {StepName} ({Elapsed}ms) - {Error}", + _flowName, prefix, name, sw.ElapsedMilliseconds, ex.Message); + throw; + } + } + + public T Step(string name, Func action, string detail = null) + { + var sw = Stopwatch.StartNew(); + var step = new FlowStep { Name = name, Detail = detail }; + try + { + var result = action(); + sw.Stop(); + step.Status = StepStatus.Success; + step.ElapsedMs = sw.ElapsedMilliseconds; + _steps.Add(step); + var prefix = GetPrefix(); + _logger.LogTrace("[FLOW:{FlowName}] {Prefix}[OK] {StepName} ({Elapsed}ms){DetailSuffix}", + _flowName, prefix, name, sw.ElapsedMilliseconds, FormatDetail(detail)); + return result; + } + catch (Exception ex) + { + sw.Stop(); + step.Status = StepStatus.Failed; + step.ElapsedMs = sw.ElapsedMilliseconds; + step.ErrorMessage = ex.Message; + _steps.Add(step); + var prefix = GetPrefix(); + _logger.LogTrace("[FLOW:{FlowName}] {Prefix}[FAIL] {StepName} ({Elapsed}ms) - {Error}", + _flowName, prefix, name, sw.ElapsedMilliseconds, ex.Message); + throw; + } + } + + public void Fail(string name, string reason) + { + var step = new FlowStep { Name = name, Status = StepStatus.Failed, ErrorMessage = reason }; + _steps.Add(step); + var prefix = GetPrefix(); + _logger.LogTrace("[FLOW:{FlowName}] {Prefix}[FAIL] {StepName} - {Reason}", _flowName, prefix, name, reason); + } + + public void Skip(string name, string reason) + { + var step = new FlowStep { Name = name, Status = StepStatus.Skipped, Detail = reason }; + _steps.Add(step); + var prefix = GetPrefix(); + _logger.LogTrace("[FLOW:{FlowName}] {Prefix}[SKIP] {StepName} - {Reason}", _flowName, prefix, name, reason); + } + + public void Branch(string name) + { + _branchStack.Push(name); + var prefix = GetPrefix(); + _logger.LogTrace("[FLOW:{FlowName}] {Prefix}>> {BranchName}", _flowName, prefix, name); + } + + public void EndBranch() + { + if (_branchStack.Count > 0) + { + var name = _branchStack.Pop(); + var prefix = GetPrefix(); + _logger.LogTrace("[FLOW:{FlowName}] {Prefix}<< {BranchName}", _flowName, prefix, name); + } + } + + public bool HasFailures => _steps.Any(s => s.Status == StepStatus.Failed); + + public string GetSummary() + { + var hasFailures = HasFailures; + var overallStatus = hasFailures ? "FAILED" : "OK"; + var total = _steps.Count; + var succeeded = _steps.Count(s => s.Status == StepStatus.Success); + var failed = _steps.Count(s => s.Status == StepStatus.Failed); + var skipped = _steps.Count(s => s.Status == StepStatus.Skipped); + var elapsed = _overallStopwatch.ElapsedMilliseconds; + + var sb = new StringBuilder(); + sb.AppendLine($"Flow: {_flowName} [{overallStatus}] Total: {elapsed}ms"); + sb.AppendLine($"Steps: {total} total, {succeeded} ok, {failed} failed, {skipped} skipped"); + sb.AppendLine("----------------------------------------"); + foreach (var step in _steps) + { + var icon = step.Status == StepStatus.Success ? "[OK] " + : step.Status == StepStatus.Failed ? "[FAIL]" + : step.Status == StepStatus.Skipped ? "[SKIP]" + : "[...]"; + var time = step.ElapsedMs.HasValue ? $" ({step.ElapsedMs}ms)" : ""; + var detail = !string.IsNullOrEmpty(step.ErrorMessage) + ? $" - {step.ErrorMessage}" + : !string.IsNullOrEmpty(step.Detail) + ? $" - {step.Detail}" + : ""; + sb.AppendLine($" {icon} {step.Name}{time}{detail}"); + } + sb.Append("----------------------------------------"); + + return sb.ToString(); + } + + public void Dispose() + { + _overallStopwatch.Stop(); + var summary = GetSummary(); + _logger.LogTrace("[FLOW:{FlowName}] === END ===\n{Summary}", _flowName, summary); + } + + private string GetPrefix() + { + if (_branchStack.Count == 0) return ""; + return new string(' ', _branchStack.Count * 2) + "| "; + } + + private static string FormatDetail(string detail) + { + return string.IsNullOrEmpty(detail) ? "" : $" - {detail}"; + } + + private enum StepStatus + { + Success, + Failed, + Skipped, + InProgress + } + + private class FlowStep + { + public string Name { get; set; } + public string Detail { get; set; } + public StepStatus Status { get; set; } = StepStatus.InProgress; + public long? ElapsedMs { get; set; } + public string ErrorMessage { get; set; } + } + } +} diff --git a/GcpCertManager/GcpCertManager.csproj b/GcpCertManager/GcpCertManager.csproj index 6f4a636..e28a001 100644 --- a/GcpCertManager/GcpCertManager.csproj +++ b/GcpCertManager/GcpCertManager.csproj @@ -2,7 +2,7 @@ true - net6.0;net8.0 + net6.0;net8.0;net10.0 true disable @@ -20,13 +20,19 @@ + - + - - + + Always diff --git a/GcpCertManager/Jobs/Discovery.cs b/GcpCertManager/Jobs/Discovery.cs new file mode 100644 index 0000000..966c92e --- /dev/null +++ b/GcpCertManager/Jobs/Discovery.cs @@ -0,0 +1,222 @@ +// Copyright 2026 Keyfactor +// Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. +// You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0 +// Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions +// and limitations under the License. + +using System; +using System.Collections.Generic; +using System.Linq; +using Google.Apis.CloudResourceManager.v3; +using Google.Apis.CloudResourceManager.v3.Data; +using Keyfactor.Extensions.Orchestrator.GcpCertManager.Client; +using Keyfactor.Logging; +using Keyfactor.Orchestrators.Extensions; +using Keyfactor.Orchestrators.Extensions.Interfaces; +using Microsoft.Extensions.Logging; + +namespace Keyfactor.Extensions.Orchestrator.GcpCertManager.Jobs +{ + public class Discovery : JobBase, IDiscoveryJobExtension + { + // GCP Certificate Manager exposes a "global" location plus a handful of regional + // locations. Most certs live in "global" so default discovery to that. Operators + // can override via the Discovery job's "Directories to search" field with a + // comma-separated list (e.g. "global,us-central1,europe-west1"). + private static readonly HashSet DefaultLocations = + new HashSet(StringComparer.OrdinalIgnoreCase) { "global" }; + + // Keyfactor Command's Discovery form posts the comma-separated "Directories + // to search" value into JobProperties. Try the common key names since the + // exact casing has shifted across Command versions. + private static readonly string[] DirsToSearchKeys = { "dirs", "Dirs", "directories", "Directories", "DirsToSearch" }; + + + public Discovery(IPAMSecretResolver resolver) : base(resolver) + { + Logger = LogHandler.GetClassLogger(); + } + + public string ExtensionName => "GcpCertMgr"; + + public JobResult ProcessJob(DiscoveryJobConfiguration jobConfiguration, + SubmitDiscoveryUpdate submitDiscoveryUpdate) + { + if (jobConfiguration == null) + { + Logger.LogError("ProcessJob called with null jobConfiguration."); + return FailureResult(0, "DiscoveryJobConfiguration is null."); + } + + if (submitDiscoveryUpdate == null) + { + Logger.LogError("ProcessJob called with null submitDiscoveryUpdate."); + return FailureResult(jobConfiguration.JobHistoryId, "SubmitDiscoveryUpdate delegate is null."); + } + + using (var flow = new FlowLogger(Logger, "GcpCertMgr-Discovery")) + { + try + { + Logger.MethodEntry(LogLevel.Debug); + return PerformDiscovery(jobConfiguration, submitDiscoveryUpdate, flow); + } + catch (Exception e) + { + var msg = DescribeException(e); + flow.Fail("ProcessJob", msg); + Logger.LogError(e, "Error In Discovery.ProcessJob: {ErrorMessage}", LogHandler.FlattenException(e)); + return FailureResult(jobConfiguration.JobHistoryId, + $"Unknown exception in Discovery: {msg}", flow); + } + finally + { + Logger.MethodExit(LogLevel.Debug); + } + } + } + + private JobResult PerformDiscovery(DiscoveryJobConfiguration config, + SubmitDiscoveryUpdate submitDiscovery, FlowLogger flow) + { + // ClientMachine is interpreted as the GCP Organization ID for logging / + // labeling. The actual project set returned by Search() is controlled by the + // service account's IAM bindings - the customer scopes that at the org root. + var orgIdHint = (config.ClientMachine ?? string.Empty).Trim(); + flow.Step("ParseConfig", $"orgIdHint={(string.IsNullOrEmpty(orgIdHint) ? "" : orgIdHint)}"); + + var (locations, locationSource) = ResolveLocations(config); + flow.Step("ResolveLocations", + $"source={locationSource}, locations=[{string.Join(",", locations)}]"); + + // Authentication uses Application Default Credentials. The Keyfactor Command + // discovery-job UI does not surface store-type custom properties, so the + // file-based ServiceAccountKey custom property used by Inventory/Management + // (deprecated as of v1.2) was never reachable here in the first place. + CloudResourceManagerService crm = null; + flow.Step("CreateApiClient", () => + { + crm = new GcpCertificateManagerClient().GetCloudResourceManager(null); + }, "source=ADC"); + + List projects = null; + flow.Step("ListProjects", () => + { + projects = ListAccessibleProjects(crm, orgIdHint); + }, "filter=state=ACTIVE"); + + var activeProjects = projects ?? new List(); + flow.Step("ProjectsCount", $"count={activeProjects.Count}"); + + var discoveredLocations = new List(); + + if (activeProjects.Count == 0) + { + flow.Skip("EmitStorePaths", "no accessible projects returned"); + } + else + { + flow.Branch($"PerProject (projects={activeProjects.Count}, locations={locations.Count})"); + try + { + foreach (var project in activeProjects) + { + var projectId = project?.ProjectId; + if (string.IsNullOrWhiteSpace(projectId)) + { + flow.Skip("Project", "missing projectId"); + continue; + } + + foreach (var location in locations) + { + // Canonical GCP resource name. Operators approving the discovered + // store will need to set the store's ClientMachine to {projectId} + // and the Location custom property to {location} - documented in + // docsource/gcpcertmgr.md. + var storePath = $"projects/{projectId}/locations/{location}"; + discoveredLocations.Add(storePath); + flow.Step($"Discovered-{storePath}"); + } + } + } + finally + { + flow.EndBranch(); + } + } + + flow.Step("SubmitDiscovery", () => submitDiscovery.Invoke(discoveredLocations), + $"locationCount={discoveredLocations.Count}"); + + flow.Step("Result", $"SUCCESS - {discoveredLocations.Count} locations discovered"); + return SuccessResult(config.JobHistoryId, flow.GetSummary()); + } + + private static (HashSet Locations, string Source) ResolveLocations(DiscoveryJobConfiguration config) + { + if (config?.JobProperties != null) + { + foreach (var key in DirsToSearchKeys) + { + if (!config.JobProperties.TryGetValue(key, out var raw)) continue; + var s = raw?.ToString(); + if (string.IsNullOrWhiteSpace(s)) continue; + + var locations = new HashSet( + s.Split(new[] { ',', ';' }, StringSplitOptions.RemoveEmptyEntries) + .Select(d => d.Trim().Trim('/')) + .Where(d => d.Length > 0), + StringComparer.OrdinalIgnoreCase); + + if (locations.Count > 0) + return (locations, $"user (key={key})"); + } + } + + return (DefaultLocations, "default"); + } + + private List ListAccessibleProjects(CloudResourceManagerService crm, string orgIdHint) + { + // Projects.Search() returns every active project the calling identity can + // see across the organization, including those nested in folders. The + // customer's service account is permissioned at the org root so this is the + // correct boundary - tightening (or loosening) is done in IAM, not here. + // + // orgIdHint is logged but not used as a query filter: the v3 query syntax + // `parent:organizations/{id}` only matches direct children, missing every + // project that lives under a folder. Filtering by parent ancestry from the + // client side requires N additional GetAncestry calls per project, which + // doesn't scale and isn't necessary when IAM already constrains the result. + var ids = new List(); + string nextPageToken = null; + do + { + var req = crm.Projects.Search(); + req.Query = "state:ACTIVE"; + req.PageSize = 100; + if (!string.IsNullOrEmpty(nextPageToken)) req.PageToken = nextPageToken; + + var resp = req.Execute(); + if (resp?.Projects != null) + { + ids.AddRange(resp.Projects.Where(p => + p != null && + !string.IsNullOrWhiteSpace(p.ProjectId) && + string.Equals(p.State, "ACTIVE", StringComparison.OrdinalIgnoreCase))); + } + nextPageToken = resp?.NextPageToken; + } while (!string.IsNullOrEmpty(nextPageToken)); + + if (!string.IsNullOrEmpty(orgIdHint)) + { + Logger.LogTrace("Discovery returned {Count} accessible projects (orgIdHint={OrgIdHint}; not used as a query filter, see ListAccessibleProjects).", + ids.Count, orgIdHint); + } + + return ids; + } + } +} diff --git a/GcpCertManager/Jobs/Inventory.cs b/GcpCertManager/Jobs/Inventory.cs index b3d6165..7d47293 100644 --- a/GcpCertManager/Jobs/Inventory.cs +++ b/GcpCertManager/Jobs/Inventory.cs @@ -8,184 +8,195 @@ using System.Collections.Generic; using System.Linq; using System.Text; -using Google; using Google.Apis.CertificateManager.v1; +using Google.Apis.CertificateManager.v1.Data; using Keyfactor.Extensions.Orchestrator.GcpCertManager.Client; using Keyfactor.Logging; using Keyfactor.Orchestrators.Common.Enums; using Keyfactor.Orchestrators.Extensions; +using Keyfactor.Orchestrators.Extensions.Interfaces; using Microsoft.Extensions.Logging; using Newtonsoft.Json; namespace Keyfactor.Extensions.Orchestrator.GcpCertManager.Jobs { - public class Inventory : IInventoryJobExtension + public class Inventory : JobBase, IInventoryJobExtension { - private readonly ILogger _logger; - - public Inventory(ILogger logger) + public Inventory(IPAMSecretResolver resolver) : base(resolver) { - _logger = logger; + Logger = LogHandler.GetClassLogger(); } - public string ExtensionName => ""; + public string ExtensionName => "GcpCertMgr"; public JobResult ProcessJob(InventoryJobConfiguration jobConfiguration, SubmitInventoryUpdate submitInventoryUpdate) { - try + if (jobConfiguration == null) { - _logger.MethodEntry(); - return PerformInventory(jobConfiguration, submitInventoryUpdate); + Logger.LogError("ProcessJob called with null jobConfiguration."); + return FailureResult(0, "InventoryJobConfiguration is null."); } - catch (Exception e) + + if (submitInventoryUpdate == null) { - _logger.LogError($"Error occured in Inventory.ProcessJob: {LogHandler.FlattenException(e)}"); - throw; + Logger.LogError("ProcessJob called with null submitInventoryUpdate."); + return FailureResult(jobConfiguration.JobHistoryId, "SubmitInventoryUpdate delegate is null."); + } + + using (var flow = new FlowLogger(Logger, "GcpCertMgr-Inventory")) + { + try + { + Logger.MethodEntry(LogLevel.Debug); + return PerformInventory(jobConfiguration, submitInventoryUpdate, flow); + } + catch (Exception e) + { + var msg = DescribeException(e); + flow.Fail("ProcessJob", msg); + Logger.LogError(e, "Error in Inventory.ProcessJob: {ErrorMessage}", LogHandler.FlattenException(e)); + return FailureResult(jobConfiguration.JobHistoryId, + $"Inventory failed: {msg}", flow); + } + finally + { + Logger.MethodExit(LogLevel.Debug); + } } } - private JobResult PerformInventory(InventoryJobConfiguration config, SubmitInventoryUpdate submitInventory) + private JobResult PerformInventory(InventoryJobConfiguration config, + SubmitInventoryUpdate submitInventory, FlowLogger flow) { - try + StoreProperties storeProperties = null; + flow.Step("ParseStoreProperties", () => { - _logger.MethodEntry(LogLevel.Debug); - - StoreProperties storeProperties = JsonConvert.DeserializeObject(config.CertificateStoreDetails.Properties, - new JsonSerializerSettings {DefaultValueHandling = DefaultValueHandling.Populate}); + storeProperties = JsonConvert.DeserializeObject( + config.CertificateStoreDetails.Properties, + new JsonSerializerSettings { DefaultValueHandling = DefaultValueHandling.Populate }); storeProperties.ProjectId = config.CertificateStoreDetails.ClientMachine; + }, $"projectId={config.CertificateStoreDetails.ClientMachine}"); - _logger.LogTrace($"Store Properties:"); - _logger.LogTrace($" Location: {storeProperties.Location}"); - _logger.LogTrace($" Project Id: {storeProperties.ProjectId}"); - _logger.LogTrace($" Service Account Key Path: {storeProperties.ServiceAccountKey}"); - - _logger.LogTrace("Getting Credentials from Google..."); - var svc = new GcpCertificateManagerClient().GetGoogleCredentials(storeProperties.ServiceAccountKey); - _logger.LogTrace("Got Credentials from Google"); - - var warningFlag = false; - var sb = new StringBuilder(); - sb.Append(""); - var inventoryItems = new List(); - var nextPageToken = string.Empty; - - //todo support labels - var storePath = $"projects/{storeProperties.ProjectId}/locations/{storeProperties.Location}"; + Logger.LogTrace("Store Properties:"); + Logger.LogTrace(" Location: {Location}", storeProperties.Location); + Logger.LogTrace(" Project Id: {ProjectId}", storeProperties.ProjectId); + // ServiceAccountKey is a file PATH, not a secret value, so it is fine to log. + Logger.LogTrace(" Service Account Key Path: {ServiceAccountKey}", storeProperties.ServiceAccountKey); - do + CertificateManagerService svc = null; + flow.Step("GetGoogleCredentials", () => + { + svc = new GcpCertificateManagerClient().GetGoogleCredentials(storeProperties.ServiceAccountKey); + }, $"source={(string.IsNullOrEmpty(storeProperties.ServiceAccountKey) ? "ADC" : "file")}"); + + var warningFlag = false; + var sb = new StringBuilder(); + var inventoryItems = new List(); + var nextPageToken = string.Empty; + var storePath = ResolveGcpResourcePath( + config.CertificateStoreDetails.StorePath, + storeProperties.ProjectId, + storeProperties.Location); + + flow.Step("StorePathResolved", $"storePath={storePath}"); + + var pageCount = 0; + do + { + pageCount++; + ListCertificatesResponse certificatesResponse = null; + var token = nextPageToken; + flow.Step($"ListCertificates-page{pageCount}", () => { - var certificatesRequest = - svc.Projects.Locations.Certificates.List(storePath); + var certificatesRequest = svc.Projects.Locations.Certificates.List(storePath); certificatesRequest.Filter = "pemCertificate!=\"\""; certificatesRequest.PageSize = 100; - if (nextPageToken?.Length > 0) certificatesRequest.PageToken = nextPageToken; - - var certificatesResponse = certificatesRequest.Execute(); - _logger.LogTrace( - $"certificatesResponse: {JsonConvert.SerializeObject(certificatesResponse)}"); - - nextPageToken = null; - //Debug Write Certificate List Response from Google Cert Manager - if (certificatesResponse?.Certificates != null) - inventoryItems.AddRange(certificatesResponse.Certificates.Select( - c => - { - try - { - _logger.LogTrace( - $"Building Cert List Inventory Item Alias: {c.Name} Pem: {c.PemCertificate} Private Key: dummy (from PA API)"); - return BuildInventoryItem(c.Name, c.PemCertificate, - true, storePath, svc); - } - catch - { - _logger.LogWarning( - $"Could not fetch the certificate: {c?.Name} associated with description {c?.Description}."); - sb.Append( - $"Could not fetch the certificate: {c?.Name} associated with issuer {c?.Description}.{Environment.NewLine}"); - warningFlag = true; - return new CurrentInventoryItem(); - } - }).Where(acsii => acsii?.Certificates != null).ToList()); - - nextPageToken = certificatesResponse.NextPageToken; - } while (nextPageToken?.Length > 0); - - _logger.LogTrace("Submitting Inventory To Keyfactor via submitInventory.Invoke"); - submitInventory.Invoke(inventoryItems); - _logger.LogTrace("Submitted Inventory To Keyfactor via submitInventory.Invoke"); - - _logger.MethodExit(LogLevel.Debug); - if (warningFlag) + if (!string.IsNullOrEmpty(token)) certificatesRequest.PageToken = token; + + certificatesResponse = certificatesRequest.Execute(); + }); + + Logger.LogTrace("certificatesResponse: {Response}", JsonConvert.SerializeObject(certificatesResponse)); + + nextPageToken = null; + if (certificatesResponse?.Certificates != null) { - _logger.LogTrace("Found Warning"); - return new JobResult + foreach (var c in certificatesResponse.Certificates) { - Result = OrchestratorJobStatusJobResult.Warning, - JobHistoryId = config.JobHistoryId, - FailureMessage = sb.ToString() - }; + try + { + Logger.LogTrace( + "Building Cert List Inventory Item Alias: {Name} Pem: {Pem} Private Key: dummy (from PA API)", + c.Name, c.PemCertificate); + var item = BuildInventoryItem(c.Name, c.PemCertificate, true, storePath, svc, c.Scope); + if (item?.Certificates != null) + inventoryItems.Add(item); + } + catch (Exception inner) + { + Logger.LogWarning("Could not fetch the certificate: {Name} associated with description {Description}. {Error}", + c?.Name, c?.Description, DescribeException(inner)); + sb.AppendLine($"Could not fetch the certificate: {c?.Name} associated with issuer {c?.Description}."); + warningFlag = true; + } + } } - _logger.LogTrace("Return Success"); - return new JobResult - { - Result = OrchestratorJobStatusJobResult.Success, - JobHistoryId = config.JobHistoryId, - FailureMessage = sb.ToString() - }; - } - catch (GoogleApiException e) - { - var googleError = e.Error?.ErrorResponseContent + " " + LogHandler.FlattenException(e); + nextPageToken = certificatesResponse?.NextPageToken; + } while (!string.IsNullOrEmpty(nextPageToken)); - _logger.LogError($"PerformInventory Error: {LogHandler.FlattenException(e)}"); - return new JobResult - { - Result = OrchestratorJobStatusJobResult.Failure, - JobHistoryId = config.JobHistoryId, - FailureMessage = - $"Management/Add {googleError}" - }; - } - catch (Exception e) + flow.Step("SubmitInventory", () => submitInventory.Invoke(inventoryItems), + $"itemCount={inventoryItems.Count}"); + + // Per the playbook: append the flow summary on BOTH success and warning paths + // so operators reading job history can see what happened either way. + var summary = flow.GetSummary(); + + if (warningFlag) { - _logger.LogError($"PerformInventory Error: {LogHandler.FlattenException(e)}"); - throw; + flow.Step("Result", "WARNING - some certificates could not be fetched"); + return WarningResult(config.JobHistoryId, $"{sb}\n\n{summary}"); } + + flow.Step("Result", $"SUCCESS - {inventoryItems.Count} certificates"); + return SuccessResult(config.JobHistoryId, summary); } protected virtual CurrentInventoryItem BuildInventoryItem(string alias, string certPem, bool privateKey, - string storePath, CertificateManagerService svc) + string storePath, CertificateManagerService svc, string scope) { try { - _logger.MethodEntry(); - _logger.LogTrace($"Alias: {alias} Pem: {certPem} PrivateKey: {privateKey}"); + Logger.MethodEntry(); + Logger.LogTrace("Alias: {Alias} Pem: {Pem} PrivateKey: {PrivateKey} Scope: {Scope}", + alias, certPem, privateKey, scope ?? ""); - //1. Look up certificate map entries based on certificate name var certAttributes = GetCertificateAttributes(storePath); + // GCP omits the scope field from the response when it's the default. + // Normalize null/blank to "DEFAULT" here so Command's UI always shows a + // concrete value on inventoried certs - and so that renewal jobs land a + // non-null Scope in JobProperties when Keyfactor replays the entry params. + certAttributes["Scope"] = string.IsNullOrWhiteSpace(scope) ? "DEFAULT" : scope; var modAlias = alias.Split('/')[5]; - - _logger.LogTrace($"Got modAlias: {modAlias}"); + Logger.LogTrace("Got modAlias: {ModAlias}", modAlias); var acsi = new CurrentInventoryItem { Alias = modAlias, - Certificates = new[] {certPem}, + Certificates = new[] { certPem }, ItemStatus = OrchestratorInventoryItemStatus.Unknown, PrivateKeyEntry = privateKey, UseChainLevel = false, Parameters = certAttributes }; - _logger.MethodExit(); + Logger.MethodExit(); return acsi; } catch (Exception e) { - _logger.LogError($"Error Occurred in Inventory.BuildInventoryItem: {LogHandler.FlattenException(e)}"); + Logger.LogError("Error Occurred in Inventory.BuildInventoryItem: {Error}", LogHandler.FlattenException(e)); throw; } } @@ -194,24 +205,23 @@ protected Dictionary GetCertificateAttributes(string storePath) { try { - _logger.MethodEntry(); - _logger.LogTrace($"Store Path: {storePath}"); + Logger.MethodEntry(); + Logger.LogTrace("Store Path: {StorePath}", storePath); var locationName = storePath.Split('/')[3]; var siteSettingsDict = new Dictionary { - {"Location", locationName} + { "Location", locationName } }; - _logger.MethodExit(); + Logger.MethodExit(); return siteSettingsDict; } catch (Exception e) { - _logger.LogError( - $"Error Occurred in Inventory.GetCertificateAttributes: {LogHandler.FlattenException(e)}"); + Logger.LogError("Error Occurred in Inventory.GetCertificateAttributes: {Error}", LogHandler.FlattenException(e)); throw; } } } -} \ No newline at end of file +} diff --git a/GcpCertManager/Jobs/JobBase.cs b/GcpCertManager/Jobs/JobBase.cs new file mode 100644 index 0000000..94686f0 --- /dev/null +++ b/GcpCertManager/Jobs/JobBase.cs @@ -0,0 +1,268 @@ +// Copyright 2026 Keyfactor +// +// Licensed under the Apache License, Version 2.0 (the "License"); +// you may not use this file except in compliance with the License. +// You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, software +// distributed under the License is distributed on an "AS IS" BASIS, +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +// See the License for the specific language governing permissions and +// limitations under the License. + +using System; +using System.Text.RegularExpressions; +using Google; +using Keyfactor.Orchestrators.Common.Enums; +using Keyfactor.Orchestrators.Extensions; +using Keyfactor.Orchestrators.Extensions.Interfaces; +using Microsoft.Extensions.Logging; + +namespace Keyfactor.Extensions.Orchestrator.GcpCertManager.Jobs +{ + /// + /// Shared plumbing for GCP Certificate Manager orchestrator jobs. Provides PAM + /// resolution with warn-on-empty fallback, JobResult helpers that append the + /// summary, and exception unwrapping that surfaces + /// details (HTTP status + response body). + /// + public abstract class JobBase + { + protected ILogger Logger; + protected readonly IPAMSecretResolver Resolver; + + protected JobBase(IPAMSecretResolver resolver) + { + Resolver = resolver ?? throw new ArgumentNullException(nameof(resolver)); + } + + /// + /// Resolves a PAM-eligible field. Returns the value as-is when it is null/empty + /// (with a warning), otherwise hands it to the PAM resolver. This avoids passing + /// empty strings into PAM providers which often misinterpret them as keys. + /// + protected string ResolvePamField(string name, string value) + { + Logger.LogTrace("Attempting to resolve PAM eligible field {FieldName}", name); + if (string.IsNullOrWhiteSpace(value)) + { + Logger.LogWarning("PAM field {FieldName} has a null/empty value, returning as-is.", name); + return value; + } + return Resolver.Resolve(value); + } + + protected static JobResult SuccessResult(long jobHistoryId, string message = "") + { + return new JobResult + { + Result = OrchestratorJobStatusJobResult.Success, + JobHistoryId = jobHistoryId, + FailureMessage = message ?? "" + }; + } + + protected static JobResult WarningResult(long jobHistoryId, string message) + { + return new JobResult + { + Result = OrchestratorJobStatusJobResult.Warning, + JobHistoryId = jobHistoryId, + FailureMessage = message ?? "" + }; + } + + protected static JobResult FailureResult(long jobHistoryId, string message, FlowLogger flow = null) + { + var combined = message ?? "Unknown error"; + if (flow != null) + { + combined = $"{combined}\n\n{flow.GetSummary()}"; + } + return new JobResult + { + Result = OrchestratorJobStatusJobResult.Failure, + JobHistoryId = jobHistoryId, + FailureMessage = combined + }; + } + + /// + /// Unwraps an exception chain and produces a human-readable description. When a + /// is anywhere in the chain (including inside an + /// ), prefer its HTTP status + error response + /// content over the generic .Message - operators need to see what GCP + /// actually returned (quota errors, IAM denials, malformed certs, etc). + /// + protected static string DescribeException(Exception ex) + { + if (ex == null) return "Unknown error"; + + var apiEx = FindGoogleApiException(ex); + if (apiEx != null) + { + var body = string.IsNullOrWhiteSpace(apiEx.Error?.ErrorResponseContent) + ? string.Empty + : $" - body: {Trim(apiEx.Error.ErrorResponseContent, 500)}"; + return $"GCP API error: HTTP {(int)apiEx.HttpStatusCode} {apiEx.HttpStatusCode}{body}"; + } + + if (ex is AggregateException agg && agg.InnerExceptions.Count > 0) + { + return agg.InnerExceptions[0].Message; + } + + return ex.InnerException?.Message ?? ex.Message; + } + + private static GoogleApiException FindGoogleApiException(Exception ex) + { + for (var cur = ex; cur != null; cur = cur.InnerException) + { + if (cur is GoogleApiException g) return g; + if (cur is AggregateException agg) + { + foreach (var inner in agg.InnerExceptions) + { + var found = FindGoogleApiException(inner); + if (found != null) return found; + } + } + } + return null; + } + + private static string Trim(string s, int maxLen) + { + if (string.IsNullOrEmpty(s)) return s; + if (s.Length <= maxLen) return s; + return s.Substring(0, maxLen) + "..."; + } + + /// + /// Resolve the GCP resource path (projects/{projectId}/locations/{location}) + /// for a certificate store. As of v1.2 the canonical source is the store's + /// StorePath; both Discovery-approved and manually-created stores set it + /// to projects/{projectId}/locations/{location}. The fallback to + /// ClientMachine + the Location custom property exists only to keep v1.1-shape + /// stores (where StorePath is blank or n/a) working through an upgrade, + /// and it logs a deprecation warning when it fires. + /// + protected string ResolveGcpResourcePath(string storePath, string projectId, string location) + { + if (!string.IsNullOrWhiteSpace(storePath)) + { + var trimmed = storePath.Trim(); + // Reject the historical "n/a" placeholder before pattern-matching. + if (!string.Equals(trimmed, "n/a", StringComparison.OrdinalIgnoreCase) && + trimmed.StartsWith("projects/", StringComparison.OrdinalIgnoreCase) && + trimmed.IndexOf("/locations/", StringComparison.OrdinalIgnoreCase) > 0) + { + return trimmed; + } + } + + // v1.1 fallback. Log a deprecation warning so operators reading orchestrator + // logs (or running this in dev) know the store should be migrated to the v1.2 + // schema (set StorePath to projects/{projectId}/locations/{location}). + Logger?.LogWarning( + "Store is using v1.1-shape configuration (ClientMachine={ProjectId}, Location={Location}, StorePath blank or 'n/a'). " + + "This is deprecated as of v1.2 and the fallback will be removed in v2.0. " + + "Edit the store and set Store Path to 'projects/{ProjectId}/locations/{Location}' to migrate.", + projectId, location, projectId, location); + + return $"projects/{projectId}/locations/{location}"; + } + + // GCP Certificate Manager certificate IDs must match Google's resource-id rule: + // [a-z]([-a-z0-9]*[a-z0-9])? length 1..63 + // i.e. lowercase letter first, lowercase letters/digits/hyphens after, must not + // end with a hyphen. The API rejects anything else with HTTP 400 INVALID_ARGUMENT. + // Source: https://cloud.google.com/certificate-manager/docs/reference/rest/v1/projects.locations.certificates/create#path-parameters + private static readonly Regex GcpCertificateIdPattern = new Regex( + @"^[a-z]([-a-z0-9]{0,61}[a-z0-9])?$", + RegexOptions.Compiled | RegexOptions.CultureInvariant); + + /// + /// Throws when is not a + /// legal GCP Certificate Manager resource ID. Call this before doing any + /// expensive PFX parsing or API work so the operator sees a clear error in the + /// flow trace instead of a 400 wall-of-JSON from GCP. + /// + protected static void ValidateGcpCertificateId(string alias) + { + if (string.IsNullOrWhiteSpace(alias)) + throw new ArgumentException("Certificate alias is required.", nameof(alias)); + + if (alias.Length > 63) + throw new ArgumentException( + $"GCP Certificate Manager requires the alias to be 63 characters or fewer; got '{alias}' ({alias.Length} chars).", + nameof(alias)); + + if (!GcpCertificateIdPattern.IsMatch(alias)) + { + var suggestion = SuggestValidAlias(alias); + throw new ArgumentException( + $"GCP Certificate Manager rejects the alias '{alias}'. Aliases must match [a-z]([-a-z0-9]*[a-z0-9])? - " + + $"start with a lowercase letter, contain only lowercase letters/digits/hyphens, and not end with a hyphen. " + + $"Try renaming the certificate in Keyfactor Command to '{suggestion}' and retry.", + nameof(alias)); + } + } + + // GCP Certificate Manager's create-only Scope values. Anything else produces an + // HTTP 400 INVALID_ARGUMENT from the create call, so validate up front and reject + // typos with a clear message instead of letting them reach the API. + // Source: https://cloud.google.com/certificate-manager/docs/reference/rest/v1/projects.locations.certificates#Certificate.Scope + private static readonly System.Collections.Generic.HashSet AllowedScopes = + new System.Collections.Generic.HashSet(StringComparer.Ordinal) + { + "DEFAULT", + "EDGE_CACHE", + "ALL_REGIONS", + "CLIENT_AUTH" + }; + + /// + /// Normalize the per-store Scope custom property to a value GCP will + /// accept. Blank → DEFAULT (matches pre-v1.2 behavior, so unmigrated + /// stores keep working). Other values are uppercased and validated against the + /// set GCP allows; an unknown value throws so + /// the operator sees a clear failure before any API call. + /// + protected static string ResolveScope(string configuredScope) + { + if (string.IsNullOrWhiteSpace(configuredScope)) return "DEFAULT"; + + var normalized = configuredScope.Trim().ToUpperInvariant(); + if (!AllowedScopes.Contains(normalized)) + { + throw new ArgumentException( + $"Unsupported Scope '{configuredScope}'. GCP Certificate Manager accepts only " + + "DEFAULT, EDGE_CACHE, ALL_REGIONS, or CLIENT_AUTH. Edit the store's Scope custom property and retry.", + nameof(configuredScope)); + } + return normalized; + } + + private static string SuggestValidAlias(string alias) + { + if (string.IsNullOrEmpty(alias)) return "cert"; + // Best-effort lowercase + replace illegal chars with '-' + trim leading non-letters and trailing hyphens. + var lowered = alias.ToLowerInvariant(); + var chars = new System.Text.StringBuilder(lowered.Length); + foreach (var c in lowered) + { + if ((c >= 'a' && c <= 'z') || (c >= '0' && c <= '9') || c == '-') chars.Append(c); + else chars.Append('-'); + } + var s = chars.ToString().Trim('-'); + // Resource IDs must start with a letter. + while (s.Length > 0 && !(s[0] >= 'a' && s[0] <= 'z')) s = s.Substring(1); + if (s.Length == 0) return "cert"; + return s.Length > 63 ? s.Substring(0, 63).TrimEnd('-') : s; + } + } +} diff --git a/GcpCertManager/Jobs/Management.cs b/GcpCertManager/Jobs/Management.cs index 29da573..b37d27e 100644 --- a/GcpCertManager/Jobs/Management.cs +++ b/GcpCertManager/Jobs/Management.cs @@ -1,33 +1,30 @@ -// Copyright 2025 Keyfactor +// Copyright 2025 Keyfactor // Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. // You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0 // Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, // WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions // and limitations under the License. using System; -using System.Collections.Generic; -using System.Drawing; using System.IO; using System.Linq; using System.Text; -using Google; +using System.Threading; using Google.Apis.CertificateManager.v1; using Google.Apis.CertificateManager.v1.Data; using Keyfactor.Extensions.Orchestrator.GcpCertManager.Client; using Keyfactor.Logging; using Keyfactor.Orchestrators.Common.Enums; using Keyfactor.Orchestrators.Extensions; +using Keyfactor.Orchestrators.Extensions.Interfaces; using Microsoft.Extensions.Logging; using Newtonsoft.Json; using Org.BouncyCastle.Crypto; using Org.BouncyCastle.OpenSsl; using Org.BouncyCastle.Pkcs; -using Org.BouncyCastle.X509; -using static Org.BouncyCastle.Math.EC.ECCurve; namespace Keyfactor.Extensions.Orchestrator.GcpCertManager.Jobs { - public class Management : IManagementJobExtension + public class Management : JobBase, IManagementJobExtension { private static readonly string certStart = "-----BEGIN CERTIFICATE-----\n"; private static readonly string certEnd = "\n-----END CERTIFICATE-----"; @@ -38,433 +35,314 @@ public class Management : IManagementJobExtension private static readonly Func Pemify = ss => ss.Length <= 64 ? ss : ss.Substring(0, 64) + "\n" + Pemify(ss.Substring(64)); - private readonly ILogger _logger; - - public Management(ILogger logger) - { - _logger = logger; - } - protected internal virtual AsymmetricKeyEntry KeyEntry { get; set; } - protected internal string CertificateName { get; set; } - public string ExtensionName => ""; + public Management(IPAMSecretResolver resolver) : base(resolver) + { + Logger = LogHandler.GetClassLogger(); + } + public string ExtensionName => "GcpCertMgr"; public JobResult ProcessJob(ManagementJobConfiguration jobConfiguration) { - try + if (jobConfiguration == null) { - _logger.MethodEntry(LogLevel.Debug); - - return PerformManagement(jobConfiguration); + Logger.LogError("ProcessJob called with null jobConfiguration."); + return FailureResult(0, "ManagementJobConfiguration is null."); } - catch (Exception e) + + using (var flow = new FlowLogger(Logger, "GcpCertMgr-Management")) { - _logger.LogError($"Error Occurred in Management.ProcessJob: {LogHandler.FlattenException(e)}"); - throw; + try + { + Logger.MethodEntry(LogLevel.Debug); + return PerformManagement(jobConfiguration, flow); + } + catch (Exception e) + { + var msg = DescribeException(e); + flow.Fail("ProcessJob", msg); + Logger.LogError(e, "Error in Management.ProcessJob: {ErrorMessage}", LogHandler.FlattenException(e)); + return FailureResult(jobConfiguration.JobHistoryId, + $"Management failed: {msg}", flow); + } + finally + { + Logger.MethodExit(LogLevel.Debug); + } } } - private JobResult PerformManagement(ManagementJobConfiguration config) + private JobResult PerformManagement(ManagementJobConfiguration config, FlowLogger flow) { - try + StoreProperties storeProperties = null; + flow.Step("ParseStoreProperties", () => { - _logger.MethodEntry(); - - StoreProperties storeProperties = JsonConvert.DeserializeObject(config.CertificateStoreDetails.Properties, + storeProperties = JsonConvert.DeserializeObject( + config.CertificateStoreDetails.Properties, new JsonSerializerSettings { DefaultValueHandling = DefaultValueHandling.Populate }); storeProperties.ProjectId = config.CertificateStoreDetails.ClientMachine; + }, $"projectId={config.CertificateStoreDetails.ClientMachine}"); - _logger.LogTrace($"Store Properties:"); - _logger.LogTrace($" Location: {storeProperties.Location}"); - _logger.LogTrace($" Project Id: {storeProperties.ProjectId}"); - _logger.LogTrace($" Service Account Key Path: {storeProperties.ServiceAccountKey}"); - - _logger.LogTrace("Getting Credentials from Google..."); - var svc = new GcpCertificateManagerClient().GetGoogleCredentials(storeProperties.ServiceAccountKey); - _logger.LogTrace("Got Credentials from Google"); - - var storePath = $"projects/{storeProperties.ProjectId}/locations/{storeProperties.Location}"; - CertificateName = config.JobCertificate.Alias; + Logger.LogTrace("Store Properties:"); + Logger.LogTrace(" Location: {Location}", storeProperties.Location); + Logger.LogTrace(" Project Id: {ProjectId}", storeProperties.ProjectId); + Logger.LogTrace(" Service Account Key Path: {ServiceAccountKey}", storeProperties.ServiceAccountKey); - var complete = new JobResult - { - Result = OrchestratorJobStatusJobResult.Failure, - JobHistoryId = config.JobHistoryId, - FailureMessage = - "Invalid Management Operation" - }; + CertificateManagerService svc = null; + flow.Step("GetGoogleCredentials", () => + { + svc = new GcpCertificateManagerClient().GetGoogleCredentials(storeProperties.ServiceAccountKey); + }, $"source={(string.IsNullOrEmpty(storeProperties.ServiceAccountKey) ? "ADC" : "file")}"); - switch (config.OperationType) - { - case CertStoreOperationType.Add: - _logger.LogTrace("Adding..."); - complete = PerformAddition(svc, config, storePath); - break; - case CertStoreOperationType.Remove: - _logger.LogTrace("Removing..."); - complete = PerformRemoval(svc, config, storePath); - break; - default: - return complete; - } + var storePath = ResolveGcpResourcePath( + config.CertificateStoreDetails.StorePath, + storeProperties.ProjectId, + storeProperties.Location); + CertificateName = config.JobCertificate.Alias; + flow.Step("StorePathResolved", $"storePath={storePath}, alias={CertificateName}"); - _logger.MethodExit(); - return complete; - } - catch (GoogleApiException e) - { - var googleError = e.Error?.ErrorResponseContent + " " + LogHandler.FlattenException(e); - return new JobResult - { - Result = OrchestratorJobStatusJobResult.Failure, - JobHistoryId = config.JobHistoryId, - FailureMessage = - $"Management {googleError}" - }; - } - catch (Exception e) + switch (config.OperationType) { - _logger.LogError($"Error Occurred in Management.PerformManagement: {LogHandler.FlattenException(e)}"); - throw; + case CertStoreOperationType.Add: + flow.Branch("Add"); + try { return PerformAddition(svc, config, storePath, flow); } + finally { flow.EndBranch(); } + case CertStoreOperationType.Remove: + flow.Branch("Remove"); + try { return PerformRemoval(svc, config, storePath, flow); } + finally { flow.EndBranch(); } + default: + flow.Fail("OperationType", $"unsupported: {config.OperationType}"); + return FailureResult(config.JobHistoryId, "Invalid Management Operation", flow); } } + private JobResult PerformRemoval(CertificateManagerService svc, ManagementJobConfiguration config, + string storePath, FlowLogger flow) + { + flow.Step("DeleteCertificate", () => DeleteCertificate(CertificateName, svc, storePath, flow)); + flow.Step("Result", "SUCCESS - certificate removed"); + return SuccessResult(config.JobHistoryId, flow.GetSummary()); + } - private JobResult PerformRemoval(CertificateManagerService svc, ManagementJobConfiguration config, string storePath) + private JobResult PerformAddition(CertificateManagerService svc, ManagementJobConfiguration config, + string storePath, FlowLogger flow) { - try + // Validate the alias before any API calls or PFX parsing - GCP rejects + // non-conforming IDs with HTTP 400 after we've already done the expensive + // work, so failing fast saves both time and a confusing error message. + flow.Step("ValidateAlias", () => ValidateGcpCertificateId(CertificateName), + $"alias={CertificateName}"); + + // Resolve the per-entry Scope entry parameter up front. Scope is per-cert + // (not per-store) because GCP itself allows mixed-scope certs inside the same + // (project, location). The value lands in config.JobProperties from the + // Command UI dropdown; on renewals/reenrollments Keyfactor pre-fills it from + // the cert's last-known inventory Parameters, so the cert keeps its scope + // through its lifecycle without operator intervention. + string configuredScope = null; + if (config.JobProperties != null && config.JobProperties.TryGetValue("Scope", out var rawScope)) { - _logger.MethodEntry(); + configuredScope = rawScope?.ToString(); + } + string resolvedScope = null; + flow.Step("ResolveScope", () => resolvedScope = ResolveScope(configuredScope), + $"configured={configuredScope ?? ""}"); - DeleteCertificate(CertificateName, svc, storePath); + var duplicate = false; + flow.Step("CheckForDuplicate", () => duplicate = CheckForDuplicate(storePath, CertificateName, svc), + $"alias={CertificateName}"); + Logger.LogTrace("Duplicate? = {Duplicate}", duplicate); - _logger.MethodExit(); - return new JobResult - { - Result = OrchestratorJobStatusJobResult.Success, - JobHistoryId = config.JobHistoryId, - FailureMessage = "" - }; - } - catch (GoogleApiException e) + if (duplicate && !config.Overwrite) { - var googleError = e.Error?.ErrorResponseContent + " " + LogHandler.FlattenException(e); - return new JobResult - { - Result = OrchestratorJobStatusJobResult.Failure, - JobHistoryId = config.JobHistoryId, - FailureMessage = - $"Management/Remove {googleError}" - }; + flow.Fail("DuplicateGuard", + $"alias '{config.JobCertificate.Alias}' exists; overwrite flag was not set"); + return FailureResult(config.JobHistoryId, + $"Duplicate alias {config.JobCertificate.Alias} found in Google Certificate Manager. To overwrite use the overwrite flag.", + flow); } - catch (Exception e) + + if (string.IsNullOrWhiteSpace(config.JobCertificate.PrivateKeyPassword)) { - return new JobResult - { - Result = OrchestratorJobStatusJobResult.Failure, - JobHistoryId = config.JobHistoryId, - FailureMessage = $"Management/Remove: {LogHandler.FlattenException(e)}" - }; + // Existing behaviour: this orchestrator only handles PFX entries with a + // private key password. Surface this clearly rather than silently no-op. + flow.Fail("PrivateKeyPassword", "missing - this orchestrator only supports PFX entries with a private key password"); + return FailureResult(config.JobHistoryId, + "Management/Add requires a PFX certificate with a private key password.", flow); } - } - - private JobResult PerformAddition(CertificateManagerService svc, ManagementJobConfiguration config, string storePath) - { - //Temporarily only performing additions - try + if (string.IsNullOrWhiteSpace(config.JobCertificate.Alias)) { - _logger.MethodEntry(); + Logger.LogTrace("No Alias Found"); + } - var client = new GcpCertificateManagerClient(); + // Load PFX + Pkcs12Store p = null; + flow.Step("LoadPkcs12", () => + { + var pfxBytes = Convert.FromBase64String(config.JobCertificate.Contents); + using (var pfxBytesMemoryStream = new MemoryStream(pfxBytes)) + { + p = new Pkcs12Store(pfxBytesMemoryStream, + config.JobCertificate.PrivateKeyPassword.ToCharArray()); + } + }); - var duplicate = CheckForDuplicate(storePath, CertificateName, svc); - _logger.LogTrace($"Duplicate? = {duplicate}"); + Logger.LogTrace("Created Pkcs12Store containing Alias {Alias} Contains Alias is {Contains}", + config.JobCertificate.Alias, p.ContainsAlias(config.JobCertificate.Alias)); - //Check for Duplicate already in Google Certificate Manager, if there, make sure the Overwrite flag is checked before replacing - if (duplicate && config.Overwrite || !duplicate) + // Extract private key + string alias = null; + string privateKeyString = null; + flow.Step("ExtractPrivateKey", () => + { + using (var memoryStream = new MemoryStream()) + using (TextWriter streamWriter = new StreamWriter(memoryStream)) { - _logger.LogTrace("Either not a duplicate or overwrite was chosen...."); - if (!string.IsNullOrWhiteSpace(config.JobCertificate.PrivateKeyPassword)) // This is a PFX Entry - { - - if (string.IsNullOrWhiteSpace(config.JobCertificate.Alias)) - _logger.LogTrace("No Alias Found"); - - // Load PFX - var pfxBytes = Convert.FromBase64String(config.JobCertificate.Contents); - Pkcs12Store p; - using (var pfxBytesMemoryStream = new MemoryStream(pfxBytes)) - { - p = new Pkcs12Store(pfxBytesMemoryStream, - config.JobCertificate.PrivateKeyPassword.ToCharArray()); - } - - _logger.LogTrace( - $"Created Pkcs12Store containing Alias {config.JobCertificate.Alias} Contains Alias is {p.ContainsAlias(config.JobCertificate.Alias)}"); - - // Extract private key - string alias; - string privateKeyString; - using (var memoryStream = new MemoryStream()) - { - using (TextWriter streamWriter = new StreamWriter(memoryStream)) - { - _logger.LogTrace("Extracting Private Key..."); - var pemWriter = new PemWriter(streamWriter); - _logger.LogTrace("Created pemWriter..."); - alias = p.Aliases.Cast().SingleOrDefault(a => p.IsKeyEntry(a)); - _logger.LogTrace($"Alias = {alias}"); - var publicKey = p.GetCertificate(alias).Certificate.GetPublicKey(); - _logger.LogTrace($"publicKey = {publicKey}"); - KeyEntry = p.GetKey(alias); - _logger.LogTrace($"KeyEntry = {KeyEntry}"); - if (KeyEntry == null) throw new Exception("Unable to retrieve private key"); - - var privateKey = KeyEntry.Key; - var keyPair = new AsymmetricCipherKeyPair(publicKey, privateKey); - - pemWriter.WriteObject(keyPair.Private); - streamWriter.Flush(); - privateKeyString = Encoding.ASCII.GetString(memoryStream.GetBuffer()).Trim() - .Replace("\r", "").Replace("\0", ""); - memoryStream.Close(); - streamWriter.Close(); - _logger.LogTrace("Finished Extracting Private Key..."); - } - } - - var pubCertPem = - Pemify(Convert.ToBase64String(p.GetCertificate(alias).Certificate.GetEncoded())); - _logger.LogTrace($"Public cert Pem {pubCertPem}"); - - var certPem = privateKeyString + certStart + pubCertPem + certEnd; - - _logger.LogTrace($"Got certPem {certPem}"); - - pubCertPem = $"-----BEGIN CERTIFICATE-----\r\n{pubCertPem}\r\n-----END CERTIFICATE-----"; - - _logger.LogTrace($"Public Cert Pem: {pubCertPem}"); - - //Create the certificate in Google - var gCertificate = new Certificate - { - SelfManaged = new SelfManagedCertificate - { PemCertificate = pubCertPem, PemPrivateKey = privateKeyString }, - Name = CertificateName, - Description = CertificateName, - Scope = "DEFAULT" //Scope does not come back in inventory so just hard code it for now - }; - - _logger.LogTrace( - $"Created Google Certificate Object: {JsonConvert.SerializeObject(gCertificate)}"); - - if (duplicate && config.Overwrite) - ReplaceCertificate(gCertificate, svc, storePath); - else - AddCertificate(gCertificate, svc, storePath); - - _logger.MethodExit(); - - //Return success from job - return new JobResult - { - Result = OrchestratorJobStatusJobResult.Success, - JobHistoryId = config.JobHistoryId, - FailureMessage = "" - }; - } + var pemWriter = new PemWriter(streamWriter); + alias = p.Aliases.Cast().SingleOrDefault(a => p.IsKeyEntry(a)); + Logger.LogTrace("Alias = {Alias}", alias); + var publicKey = p.GetCertificate(alias).Certificate.GetPublicKey(); + KeyEntry = p.GetKey(alias); + if (KeyEntry == null) throw new Exception("Unable to retrieve private key"); + + var privateKey = KeyEntry.Key; + var keyPair = new AsymmetricCipherKeyPair(publicKey, privateKey); + + pemWriter.WriteObject(keyPair.Private); + streamWriter.Flush(); + privateKeyString = Encoding.ASCII.GetString(memoryStream.GetBuffer()).Trim() + .Replace("\r", "").Replace("\0", ""); } - _logger.MethodExit(); - return new JobResult - { - Result = OrchestratorJobStatusJobResult.Failure, - JobHistoryId = config.JobHistoryId, - FailureMessage = - $"Duplicate alias {config.JobCertificate.Alias} found in Google Certificate Manager. To overwrite use the overwrite flag." - }; - } - catch (GoogleApiException e) + }); + + var pubCertPem = Pemify(Convert.ToBase64String(p.GetCertificate(alias).Certificate.GetEncoded())); + // Don't log private key material - only the public chain + alias. + Logger.LogTrace("Public cert PEM extracted for alias {Alias}", alias); + + // Note: certPem includes the (decrypted) private key. It is intentionally NOT + // logged. The variable is retained because the legacy code computed it inline; + // the actual upload below uses pubCertPem + privateKeyString separately. + var certPem = privateKeyString + certStart + pubCertPem + certEnd; + _ = certPem; + + pubCertPem = $"-----BEGIN CERTIFICATE-----\r\n{pubCertPem}\r\n-----END CERTIFICATE-----"; + + // Build the GCP certificate object. Don't serialize+log; that would leak the + // private key into trace logs. + // + // Scope comes from the per-entry "Scope" entry parameter and is honored only + // on Add. On Replace the patch's UpdateMask is "SelfManaged", so GCP ignores + // every other field on the body (including Scope) - which is correct, since + // GCP refuses to change scope on an existing cert anyway. + var gCertificate = new Certificate { - var googleError = e.Error?.ErrorResponseContent + " " + LogHandler.FlattenException(e); - _logger.LogError($"PerformManagement Error: {LogHandler.FlattenException(e)}"); + SelfManaged = new SelfManagedCertificate { PemCertificate = pubCertPem, PemPrivateKey = privateKeyString }, + Name = CertificateName, + Description = CertificateName, + Scope = resolvedScope + }; - return new JobResult - { - Result = OrchestratorJobStatusJobResult.Failure, - JobHistoryId = config.JobHistoryId, - FailureMessage = - $"Management/Add {googleError}" - }; + if (duplicate && config.Overwrite) + { + flow.Step("ReplaceCertificate", () => ReplaceCertificate(gCertificate, svc, storePath, flow)); } - catch (Exception e) + else { - _logger.LogError($"PerformManagement Error: {LogHandler.FlattenException(e)}"); - - return new JobResult - { - Result = OrchestratorJobStatusJobResult.Failure, - JobHistoryId = config.JobHistoryId, - FailureMessage = $"Management/Add {LogHandler.FlattenException(e)}" - }; + flow.Step("AddCertificate", () => AddCertificate(gCertificate, svc, storePath, flow)); } + + flow.Step("Result", duplicate ? "SUCCESS - certificate replaced" : "SUCCESS - certificate added"); + return SuccessResult(config.JobHistoryId, flow.GetSummary()); } - private void AddCertificate(Certificate gCertificate, CertificateManagerService svc, string storePath) + private void AddCertificate(Certificate gCertificate, CertificateManagerService svc, string storePath, FlowLogger flow) { var addCertificateRequest = svc.Projects.Locations.Certificates.Create(gCertificate, storePath); addCertificateRequest.CertificateId = gCertificate.Name; var addCertificateResponse = addCertificateRequest.Execute(); - WaitForOperation(svc, addCertificateResponse.Name); - - _logger.LogTrace($"Certificate Created in Google Cert Manager with Name {addCertificateResponse.Name}"); + flow.Step("WaitForOperation-Add", () => WaitForOperation(svc, addCertificateResponse.Name), + $"operation={addCertificateResponse.Name}"); - _logger.MethodExit(); + Logger.LogTrace("Certificate Created in Google Cert Manager with Name {Name}", addCertificateResponse.Name); } - private void ReplaceCertificate(Certificate gCertificate, CertificateManagerService svc, string storePath) + private void ReplaceCertificate(Certificate gCertificate, CertificateManagerService svc, string storePath, FlowLogger flow) { - _logger.MethodEntry(); - var replaceCertificateRequest = svc.Projects.Locations.Certificates.Patch(gCertificate, storePath + $"/certificates/{CertificateName}"); replaceCertificateRequest.UpdateMask = "SelfManaged"; var replaceCertificateResponse = replaceCertificateRequest.Execute(); - WaitForOperation(svc, replaceCertificateResponse.Name); - - _logger.LogTrace($"Certificate Replaced in Google Cert Manager with Name {replaceCertificateResponse.Name}"); + flow.Step("WaitForOperation-Replace", () => WaitForOperation(svc, replaceCertificateResponse.Name), + $"operation={replaceCertificateResponse.Name}"); - _logger.MethodExit(); + Logger.LogTrace("Certificate Replaced in Google Cert Manager with Name {Name}", replaceCertificateResponse.Name); } - private void DeleteCertificate(string certificateName, - CertificateManagerService svc, string storePath) + private void DeleteCertificate(string certificateName, CertificateManagerService svc, string storePath, FlowLogger flow) { - try - { - _logger.MethodEntry(); - - var certificatesRequest = svc.Projects.Locations.Certificates.List(storePath); - certificatesRequest.Filter = $"name=\"{storePath}/certificates/{certificateName}\""; - - var certificatesResponse = certificatesRequest.Execute(); - _logger.LogTrace($"certificatesResponse Json {JsonConvert.SerializeObject(certificatesResponse)}"); + var certificatesRequest = svc.Projects.Locations.Certificates.List(storePath); + certificatesRequest.Filter = $"name=\"{storePath}/certificates/{certificateName}\""; - if (certificatesResponse?.Certificates?.Count > 0) - { - var deleteCertificateRequest = - svc.Projects.Locations.Certificates.Delete(storePath + $"/certificates/{certificateName}"); + var certificatesResponse = certificatesRequest.Execute(); + Logger.LogTrace("certificatesResponse Json {Response}", JsonConvert.SerializeObject(certificatesResponse)); - var deleteCertificateResponse = deleteCertificateRequest.Execute(); - _logger.LogTrace( - $"deleteCertificateResponse Json {JsonConvert.SerializeObject(deleteCertificateResponse)}"); - WaitForOperation(svc, deleteCertificateResponse.Name); + if (certificatesResponse?.Certificates?.Count > 0) + { + var deleteCertificateRequest = + svc.Projects.Locations.Certificates.Delete(storePath + $"/certificates/{certificateName}"); - _logger.LogTrace($"Deleted {deleteCertificateResponse.Name} Certificate During Replace Procedure"); - } - else - { - string msg = $"Certificate {certificateName} not found for {storePath}."; - _logger.LogWarning(msg); - throw new Exception(msg); - } + var deleteCertificateResponse = deleteCertificateRequest.Execute(); + Logger.LogTrace("deleteCertificateResponse Json {Response}", JsonConvert.SerializeObject(deleteCertificateResponse)); + flow.Step("WaitForOperation-Delete", () => WaitForOperation(svc, deleteCertificateResponse.Name), + $"operation={deleteCertificateResponse.Name}"); - _logger.MethodExit(); + Logger.LogTrace("Deleted {Name} Certificate", deleteCertificateResponse.Name); } - catch (Exception e) + else { - _logger.LogError($"Error occured in Management.DeleteCertificate: {LogHandler.FlattenException(e)}"); - throw; + var msg = $"Certificate {certificateName} not found for {storePath}."; + Logger.LogWarning(msg); + throw new Exception(msg); } } private bool CheckForDuplicate(string path, string alias, CertificateManagerService client) { - try - { - _logger.MethodEntry(); - var certificatesRequest = - client.Projects.Locations.Certificates.List(path); - certificatesRequest.Filter = $"name=\"{path}/certificates/{alias}\""; + var certificatesRequest = client.Projects.Locations.Certificates.List(path); + certificatesRequest.Filter = $"name=\"{path}/certificates/{alias}\""; - var certificatesResponse = certificatesRequest.Execute(); - _logger.LogTrace($"certificatesResponse Json {JsonConvert.SerializeObject(certificatesResponse)}"); + var certificatesResponse = certificatesRequest.Execute(); + Logger.LogTrace("certificatesResponse Json {Response}", JsonConvert.SerializeObject(certificatesResponse)); - if (certificatesResponse?.Certificates?.Count == 1) - { - _logger.MethodExit(); - return true; - } - - _logger.MethodExit(); - return false; - } - catch (Exception e) - { - _logger.LogError( - $"Error Checking for Duplicate Cert in Management.CheckForDuplicate {LogHandler.FlattenException(e)}"); - throw; - } + return certificatesResponse?.Certificates?.Count == 1; } private void WaitForOperation(CertificateManagerService client, string operationName) { - _logger.MethodEntry(); - - DateTime endTime = DateTime.Now.AddMilliseconds(OPERATION_MAX_WAIT_MILLISECONDS); - Operation operation = new Operation(); - ProjectsResource.LocationsResource.OperationsResource.GetRequest getRequest = client.Projects.Locations.Operations.Get(operationName); + var endTime = DateTime.Now.AddMilliseconds(OPERATION_MAX_WAIT_MILLISECONDS); + var getRequest = client.Projects.Locations.Operations.Get(operationName); while (DateTime.Now < endTime) { - _logger.LogTrace($"Attempting WAIT for {operationName} at {DateTime.Now.ToString()}."); - operation = getRequest.Execute(); + Logger.LogTrace("Attempting WAIT for {OperationName} at {Now}.", operationName, DateTime.Now); + var operation = getRequest.Execute(); if (operation.Done == true) { - _logger.LogDebug($"End WAIT for {operationName}. Task DONE."); - _logger.MethodExit(); + Logger.LogDebug("End WAIT for {OperationName}. Task DONE.", operationName); return; } - System.Threading.Thread.Sleep(OPERATION_INTERVAL_WAIT_MILLISECONDS); + Thread.Sleep(OPERATION_INTERVAL_WAIT_MILLISECONDS); } - _logger.MethodExit(); - throw new Exception($"{operationName} was still processing after the {OPERATION_MAX_WAIT_MILLISECONDS.ToString()} millisecond maximum wait time."); - } - - private string GetCommonNameFromSubject(string subject) - { - try - { - _logger.MethodEntry(); - var array1 = subject.Split(','); - foreach (var x in array1) - { - var itemArray = x.Split('='); - - switch (itemArray[0].ToUpper()) - { - case "CN": - return itemArray[1]; - } - } - - _logger.LogTrace("Could not get subject returning empty string..."); - _logger.MethodExit(); - return ""; - } - catch (Exception e) - { - _logger.LogError( - $"Error Checking for Duplicate Cert in Management.GetCommonNameFromSubject {LogHandler.FlattenException(e)}"); - throw; - } + throw new Exception($"{operationName} was still processing after the {OPERATION_MAX_WAIT_MILLISECONDS} millisecond maximum wait time."); } } -} \ No newline at end of file +} diff --git a/GcpCertManager/manifest.json b/GcpCertManager/manifest.json index e728639..c97908c 100644 --- a/GcpCertManager/manifest.json +++ b/GcpCertManager/manifest.json @@ -8,6 +8,10 @@ "CertStores.GcpCertMgr.Management": { "assemblypath": "GcpCertManager.dll", "TypeFullName": "Keyfactor.Extensions.Orchestrator.GcpCertManager.Jobs.Management" + }, + "CertStores.GcpCertMgr.Discovery": { + "assemblypath": "GcpCertManager.dll", + "TypeFullName": "Keyfactor.Extensions.Orchestrator.GcpCertManager.Jobs.Discovery" } } } diff --git a/README.md b/README.md index 6fd92f7..2796be2 100644 --- a/README.md +++ b/README.md @@ -5,9 +5,9 @@

Integration Status: production -Release -Issues -GitHub Downloads (all assets, all releases) +Release +Issues +GitHub Downloads (all assets, all releases)

@@ -33,15 +33,26 @@ The GCP Certificate Manager Orchestrator Extension remotely manages certificates on the Google Cloud Platform Certificate Manager Product. -This orchestrator extension implements three job types - Inventory, Management Add, and Management Remove. Below are the steps necessary to configure this Orchestrator Extension. It supports adding certificates with private keys only. The GCP Certificate Manager Orchestrator Extension supports the replacement of unbound certificates as well as certificates bound to existing map entries, but it does **not** support specifying map entry bindings when adding new certificates. +This orchestrator extension implements four job types - Inventory, Management Add, Management Remove, and Discovery. It supports adding certificates with private keys only. The orchestrator supports the replacement of unbound certificates as well as certificates bound to existing map entries, but it does **not** support specifying map entry bindings when adding new certificates. +### Configuration model +Every `GcpCertMgr` store identifies its target Certificate Manager instance through the canonical GCP resource path in **Store Path**: + +``` +projects/{projectId}/locations/{location} +``` + +This applies equally to manually-created stores and Discovery-approved stores. The `Location` custom property is deprecated as of v1.2 and used only as a v1.1 backwards-compatibility fallback. **Client Machine** is a display label for grouping in Command's UI - the recommended value is the GCP Organization ID. + +The Discovery job enumerates every GCP project that the orchestrator's service account can see and proposes one candidate store per (project, location) pair, with Store Path pre-populated in canonical form. The actual scope of discovery is bounded by IAM - grant the service account the appropriate role at the organization root and Discovery will return everything underneath. See the GCP Certificate Manager store-type page for full operator-facing details. ## Compatibility This integration is compatible with Keyfactor Universal Orchestrator version 10.4.1 and later. ## Support + The Google Cloud Provider Certificate Manager Universal Orchestrator extension is supported by Keyfactor. If you require support for any issues or have feature request, please open a support ticket by either contacting your Keyfactor representative or via the Keyfactor Support Portal at https://support.keyfactor.com. > If you want to contribute bug fixes or additional enhancements, use the **[Pull requests](../../pulls)** tab. @@ -50,47 +61,265 @@ The Google Cloud Provider Certificate Manager Universal Orchestrator extension i Before installing the Google Cloud Provider Certificate Manager Universal Orchestrator extension, we recommend that you install [kfutil](https://github.com/Keyfactor/kfutil). Kfutil is a command-line tool that simplifies the process of creating store types, installing extensions, and instantiating certificate stores in Keyfactor Command. +## GcpCertMgr Certificate Store Type -**Google Cloud Configuration** +To use the Google Cloud Provider Certificate Manager Universal Orchestrator extension, you **must** create the GcpCertMgr Certificate Store Type. This only needs to happen _once_ per Keyfactor Command instance. -1. Read up on [Google Certificate Manager](https://cloud.google.com/certificate-manager/docs) and how it works. +The `GcpCertMgr` store type represents a single (Project, Location) pair within Google Cloud Certificate Manager. The orchestrator manages self-managed certificates inside that container - listing them for inventory, uploading new PFX certificates, and deleting existing certificates by alias. -2. Either a Google Service Account is needed with the following permissions (Note: Workload Identity Management Should be used but at the time of the writing it was not available in the .net library yet), or the virtual machine running the Keyfactor Orchestrator Service must reside within Google Cloud. -![](docsource/images/ServiceAccountSettings.gif) +#### Configuration model (v1.2+) -3. The following Api Access is needed: -![](docsource/images/ApiAccessNeeded.gif) +Every `GcpCertMgr` store - whether Discovery-approved or manually created - identifies its target Certificate Manager instance through the **Store Path** field: -4. If authenticating via service account, download the Json Credential file as shown below: -![](docsource/images/GoogleKeyJsonDownload.gif) +``` +projects/{projectId}/locations/{location} +``` +That single value carries both the GCP project and the location (region or `global`). Inventory and Management read it directly; **Client Machine** is a display label for grouping in Command's UI and is not parsed by the orchestrator. -## GcpCertMgr Certificate Store Type +##### Field semantics -To use the Google Cloud Provider Certificate Manager Universal Orchestrator extension, you **must** create the GcpCertMgr Certificate Store Type. This only needs to happen _once_ per Keyfactor Command instance. +| Field | What it carries | Read by | +|---|---|---| +| **Store Path** | Canonical GCP resource path: `projects/{projectId}/locations/{location}`. The `{location}` segment is the GCP region (or `global`) the store targets - this is the only place the orchestrator actually reads the location from for new stores. | Inventory, Management, Discovery (emit) | +| **Client Machine** | Display label only. Recommended: GCP Organization ID (e.g. `1005564431893`). Not parsed. | UI grouping in Command | +| **Service Account Key File Path** (custom, *deprecated*) | v1.1 shape only. Leave blank for new stores - authentication uses Application Default Credentials. | Credential loader fallback; emits a deprecation warning when read | +| **Location** (custom, *deprecated*) | v1.1 shape only. New stores leave it blank. Used as a fallback when Store Path is empty or `n/a`. | v1.1 fallback path; emits a deprecation warning when read | +| **Certificate Scope** (*entry parameter*, not a store property) | GCP `scope` for an individual certificate entry. One of `DEFAULT`, `ALL_REGIONS`, `EDGE_CACHE`, `CLIENT_AUTH`. Set per-cert at Add time; Inventory persists the existing value back from GCP so renewals carry it forward. Defined under the store type's EntryParameters, not Properties. See "Certificate scope" below. | Management/Add (read from `JobProperties`), Inventory (write to per-entry `Parameters`) | + +##### Location semantics: where the GCP region lives + +GCP region names (`global`, `us-central1`, `europe-west1`, ...) appear in three distinct places across the orchestrator. They look related but they are **not interchangeable**, and only one of them is load-bearing for new stores. Operators who only skim the field semantics table often miss this and end up confused about which Location field to set where. + +| # | Where it appears | What it is | What reads it | +|---|---|---|---| +| 1 | **The `{location}` segment of `Store Path`** (e.g. the `global` in `projects/edgecerts/locations/global`) | The actual GCP region the store targets. Source of truth. | Inventory and Management both call `JobBase.ResolveGcpResourcePath` which returns Store Path verbatim when it matches the canonical form. The location segment is parsed back out (string split) when the Inventory job needs to populate the `Location` parameter on each returned certificate. | +| 2 | **The `Location` custom store property** | A v1.1-shape field. New stores leave it blank. | Only the v1.1 fallback path inside `JobBase.ResolveGcpResourcePath` - it builds `projects/{ClientMachine}/locations/{Location}` when Store Path is blank or `n/a`, and emits a `LogWarning` each time naming the migration step. Removal scheduled for v2.0. | +| 3 | **The Discovery job's "Directories to search" form field** | Operator INPUT to the discovery job. A comma-separated list of regions to enumerate, e.g. `global,us-central1,europe-west1`. | `Discovery.ResolveLocations` parses the list. Discovery then emits one candidate store path per `(project × location)` combination, where the location segment of each emitted Store Path comes from this list. The list itself does **not** propagate to the resulting stores - it has no afterlife once Discovery has emitted candidates. | + +###### How the three relate + +For a **v1.2 store created via Discovery**: + +1. Operator types `global,us-central1` into the discovery job's "Directories to search" - this is place #3. +2. Discovery emits `projects/edgecerts/locations/global` and `projects/edgecerts/locations/us-central1`. Each emitted string is consumed as place #1 (Store Path) when the candidate is approved. +3. The auto-created store has Store Path populated; the **Location custom property is blank** and unused. Place #2 has no role. +4. Inventory reads Store Path (place #1) for the GCP API path, and parses the location segment back out for the `Location` parameter on each cert item. +5. Discovery's "Directories to search" value (place #3) is gone - it never landed on the store. + +For a **v1.2 store created manually**: + +- Type the full canonical path into Store Path (place #1) - e.g. `projects/edgecerts/locations/global`. +- Leave the Location custom property (place #2) blank. + +For a **v1.1 store mid-migration**: + +- Store Path is blank or `n/a`; the orchestrator falls back to building the GCP path from `Client Machine` + `Location` (place #2). A deprecation warning is logged every job run. Migrate by populating Store Path (place #1) and clearing place #2. + +###### Quick reference + +- "Where do I tell the orchestrator which GCP region to use for *this store*?" → **the `{location}` segment of Store Path** (place #1). +- "Where do I tell Discovery which regions to enumerate across *the whole org*?" → **Directories to search** on the discovery job (place #3). +- "What's the Location custom field for?" → Nothing, unless you're maintaining a v1.1 store and haven't migrated yet (place #2). +- "Why does the inventory result still have a `Location` parameter on each certificate?" → That's parsed out of Store Path's location segment for downstream filtering in Command. It mirrors place #1, not place #2. + +##### Manually creating a store + +Set: + +- **Client Machine**: GCP Organization ID +- **Store Path**: `projects/{projectId}/locations/{location}` - e.g. `projects/edgecerts/locations/global` +- **Service Account Key File Path**: leave blank (deprecated; ADC is used) +- **Location**: leave blank + +Scope is set per-cert at Add time (entry parameter, not store property) - see "Certificate scope" below. A single store can hold certificates at multiple scopes. + +Authentication uses Application Default Credentials - see "Service account credentials" below. + +##### Approving a Discovery-discovered store + +After the discovery job runs, candidates appear in **Locations → Certificate Stores → Discover** (a new tab next to "Certificate Stores"). Tick the candidates you want to track, click **MANAGE**, and Command opens the per-candidate edit dialog. The relevant fields: + +| Field | Action | +|---|---| +| **Client Machine** | Pre-filled by Command (often the orchestrator hostname). Display label only - the orchestrator does not parse it. Leave as-is, or change to the GCP Organization ID for cleaner UI grouping. | +| **Store Path** | Pre-filled with the canonical GCP path Discovery emitted (e.g. `projects/edgecerts/locations/global`). **Don't edit** - this is what Inventory and Management read. | +| **Application** | Optional, free-form. | +| **Location** (custom) | Leave blank. Deprecated v1.1 field; the location is parsed from Store Path. | +| **Service Account Key File Path** (custom) | Leave blank. Deprecated v1.1 field; authentication uses Application Default Credentials. | +| **Create Certificate Store If Missing** | **Check this.** Tells Command to create a new certificate store record from this candidate. Without it, the candidate sits in the discover tab with no store backing it. | +| **Inventory Schedule** | Pick a cadence (e.g. Daily) for the inventory job to run after the store is created. | + +Click **SAVE** and the store is created. The next inventory run on its schedule will populate it with whatever certificates exist in that (project, location). + +#### Discovery job configuration + +Discovery is configured against the GCP Certificate Manager store type and enumerates candidate stores across an entire GCP organization. It uses the Cloud Resource Manager v3 API (`projects.search`) to list every active project the orchestrator's service account can see, then emits one candidate store path per (project, location) combination. + +The "Schedule Discovery" dialog inherits its layout from Keyfactor Command's generic Discovery UI, which was designed for filesystem-based store types (Java keystores, PEM files). Most fields don't apply to GCP. Here is what each field is and what to do with it: + +| Field on Schedule Discovery | What to put | +|---|---| +| **Category** | `GCP Certificate Manager` - already populated when reaching this dialog from the GCP store type. | +| **Orchestrator** | Select an approved orchestrator with the `GcpCertMgr` capability. | +| **Schedule** | When discovery should run. `Immediate` runs once on save; pick a recurring schedule for periodic re-enumeration. | +| **Directories to search** | **Required.** Type `global` for the default behavior of searching only GCP's global Certificate Manager location, which is what almost every operator wants. See "Should I ever put something other than `global`?" below for the rare exceptions. | +| **Directories to ignore** | Leave blank. Filesystem-store concept; not used by GCP discovery. | +| **Extensions** | Leave blank. Filesystem-store concept; not used. | +| **File name patterns to match** | Leave blank. Filesystem-store concept; not used. | +| **Follow SymLinks** | Leave unchecked. Filesystem-store concept; not used. | +| **Include PKCS12 Files?** | Leave unchecked. Filesystem-store concept; not used. | + +> **Why are most fields irrelevant to GCP?** Command's Discovery UI is one form template shared across every store type. For filesystem-based store types like Java keystores or PEM files, fields like *Directories to ignore*, *Extensions*, *File name patterns to match*, *Follow SymLinks*, and *Include PKCS12 Files?* are useful - they let the orchestrator narrow which files on disk it should treat as candidate stores. GCP Certificate Manager isn't a filesystem; the orchestrator uses Cloud Resource Manager + Certificate Manager APIs, so these fields are not consulted. The orchestrator does not raise an error if you fill them in; it just ignores them. + +##### Should I ever put something other than `global`? + +Almost never. Concrete guidance: + +- **Just type `global` → searches the `global` GCP location only.** This is the right answer for the vast majority of GCP Certificate Manager deployments, because certificates attached to GCP's *global* external Application Load Balancer (the most common load balancer in GCP) are stored in the `global` Certificate Manager location. +- **Add specific regions** (e.g. `global,us-central1,europe-west1`) only if your organization runs **regional** external Application Load Balancers, or has data-residency requirements that pin certificates to specific regions. If you're not sure whether that describes your environment, the answer is "you don't need this" and you should just type `global`. +- **Don't list every GCP region** (`us-central1,us-east1,...`). Discovery does not probe candidates - it emits one (project × location) pair regardless of whether that combination has any certs. Listing 40 regions for a 100-project org produces 4,000 candidate stores, most empty, all cluttering Command's certificate store list. + +The format is a comma-separated list of GCP location names exactly as GCP names them. `global` is the universal location; regional names follow GCP's standard `-` form (e.g. `us-central1`, `europe-west1`, `asia-southeast1`). See the [Certificate Manager supported locations list](https://cloud.google.com/certificate-manager/docs/locations) for the canonical set. + +The candidate count is always `projects × locations`, so each region you add multiplies the size of the discovery result by the number of accessible projects. + +##### Service account credentials + +The orchestrator authenticates exclusively via Application Default Credentials. Two supported deployment modes: + +- **Inside GCP** - on a GCE VM or GKE pod with the service account attached via workload identity. ADC discovers the service account from the metadata server automatically. No host configuration needed. +- **Outside GCP** - on a Windows host or on-prem Linux. Set the `GOOGLE_APPLICATION_CREDENTIALS` machine-level environment variable to the absolute path of the service account's JSON key, then restart the Keyfactor Orchestrator service so it picks up the variable. The account that runs the orchestrator service must have read access to the JSON key file. + +The legacy `Service Account Key File Path` custom store property (a JSON filename relative to the orchestrator extension directory) is **deprecated as of v1.2** because the Discovery job has no way to surface custom store properties in Keyfactor Command's discovery-job UI - so file-based auth can't be configured uniformly across all four job types. v1.1 stores with the property populated continue to work, but every job run logs a deprecation warning. The field is scheduled for removal in v2.0; new stores should leave it blank. + +The service account needs at minimum: + +- `roles/browser` at the **organization** root - for `projects.search` to see projects nested in folders. +- `roles/certificatemanager.viewer` per project (or at the org root for inheritance) - for inventory to list certificates. +- `roles/certificatemanager.editor` - for management to add/remove certificates. + +Required APIs to enable in the **service account's home project**: + +- Cloud Resource Manager API +- Certificate Manager API (also needs to be enabled in every project you actually inventory) + +#### Certificate alias rules + +GCP Certificate Manager constrains certificate resource IDs to a strict shape: + +- 1 to 63 characters +- Lowercase letters, digits, hyphens only +- Must start with a lowercase letter +- Must not end with a hyphen +- Regex: `[a-z]([-a-z0-9]*[a-z0-9])?` + +The orchestrator validates the alias against this rule **before** any API calls or PFX parsing during Management/Add. A non-conforming alias fails fast with a `[FAIL] ValidateAlias` step in the flow trace and a suggestion of a normalized alias (e.g. `Cert1` → `cert1`). Rename the certificate in Keyfactor Command to the suggested form and retry the Management/Add job. + +#### Certificate scope + +GCP Certificate Manager attaches a `scope` to every certificate that determines which load balancer / service families can consume it. The orchestrator models this as a per-entry **entry parameter** (defined under `EntryParameters` on the store type), not a store property. That matches GCP's own data model - a single (project, location) container in GCP can legitimately hold certificates with different scopes. + +| Value | What it is for | +|---|---| +| `DEFAULT` | Global external Application Load Balancers - the standard GCP load balancer for internet-facing traffic. This is the GCP default and the right answer for most certs. | +| `ALL_REGIONS` | Cross-region **internal** Application Load Balancers. Use this when the consuming load balancer is regional/internal and replicated across regions. | +| `EDGE_CACHE` | Google Cloud Media CDN edge-cache certs. | +| `CLIENT_AUTH` | Certificates used by mTLS trust configs, or server certificates that are authorized for client authentication. | +##### How operators set it +On Management/Add (uploading a new cert into a store), Keyfactor Command renders a dropdown for "Certificate Scope" sourced from the store type's `EntryParameters` definition. Pick one of the four values; default is `DEFAULT`. The chosen value lands in `ManagementJobConfiguration.JobProperties["Scope"]` and the orchestrator passes it to GCP on the `certificates.create` call. +On renewals / reenrollments, Keyfactor pre-fills the dropdown from the certificate's last-known inventory parameters, so the cert keeps its scope automatically across renewal cycles without operator action. +##### Immutability +The `scope` field is **create-only** in the GCP API. Once GCP creates a certificate with a given scope, that scope cannot be changed by any patch operation. The orchestrator's Management/Replace path uses `UpdateMask = "SelfManaged"`, so re-adding a certificate over an existing one preserves its original scope - even if the operator selects a different scope in the renewal dialog. To migrate a certificate to a different scope, delete it (Management/Remove) and re-add it with the new scope. +##### Inventory persists scope per-cert +The Inventory job reads the `scope` field off each `certificates.list` response and writes it into the cert's `CurrentInventoryItem.Parameters` dict. GCP omits the field from the response when the cert is at the default scope; the orchestrator normalizes null/blank to `DEFAULT` so Command's UI always shows a concrete value. After the first inventory run on a store, every cert displays its actual GCP scope in Command, and that value flows back through to Management jobs on subsequent renew/reenrollment operations. + +##### What happened before v1.2.1 + +Prior to v1.2.1 the orchestrator hard-coded `Scope = "DEFAULT"` on every certificate it created and never read `scope` back during Inventory. Customers who needed non-default scopes (typically `ALL_REGIONS` for cross-region internal ALBs) had to pre-create empty placeholder certificate resources in GCP via Terraform with `scope = "ALL_REGIONS"`, then point Keyfactor at the existing resource as a Replace target. The new entry parameter removes that workaround. + +##### How the orchestrator validates the value + +`JobBase.ResolveScope` runs as the `ResolveScope` flow step on every Management/Add. It trims and uppercases the configured value, then validates it against the set GCP accepts. An unsupported value (typo, lowercase letters that don't normalize to a valid token, anything outside the four allowed values) fails with `[FAIL] ResolveScope` and a clear message naming the four legal values. Blank or null resolves to `DEFAULT`. The dropdown UI in Command should prevent invalid values from reaching the orchestrator in the first place; `ResolveScope` is defence-in-depth for direct-API submissions. + +##### Quick reference + +- "Where do I see what scope a certificate ended up with?" → Run inventory once - it surfaces in the cert's Parameters in Command. Also `gcloud certificate-manager certificates describe --location= --project=`. +- "Can I change a certificate's scope?" → No. Delete and re-add. +- "I have one store with DEFAULT certs and I want to add an ALL_REGIONS cert" → Add the new cert with Scope = `ALL_REGIONS` from the dropdown. Existing DEFAULT certs in the same store are unaffected; the field is per-entry, not store-wide. + +#### Architecture and logging + +Every job (Discovery, Inventory, Management) uses a shared `FlowLogger` to record step-by-step progress with timing. The flow summary is appended to `JobResult.FailureMessage` on **both** success and failure paths so operators reading job history can see what happened without having to pull orchestrator-side trace logs. Errors arising from the GCP SDK are unwrapped through `AggregateException` walls and reported with HTTP status + the GCP error response body, so quota errors / IAM denials / malformed certificates surface clearly in Command's UI. + +#### Migrating v1.1 stores + +A v1.1-shape store has `Store Path` empty or `n/a`, `Client Machine` set to the GCP Project ID, the `Location` custom property set to the region, and possibly the `Service Account Key File Path` custom property pointing at a JSON key in the orchestrator extension directory. These continue to work in v1.2 through fallback paths, but every inventory/management run logs deprecation warnings naming the store. To migrate, edit each affected store: + +1. Set **Store Path** to `projects/{the-current-Client-Machine-value}/locations/{the-current-Location-value}`. +2. Optionally change **Client Machine** to the GCP Organization ID for cleaner UI grouping. +3. Optionally clear the **Location** field (no longer required). +4. Configure ADC on the orchestrator host (see "Service account credentials") and clear the **Service Account Key File Path** field. +5. Save. + +The deprecation warnings will stop on the next job run once the store is fully migrated. Both fallbacks will be removed in v2.0. + +#### Design rationale: why Store Path is the source of truth + +In v1.1 the orchestrator built the GCP resource path from **Client Machine** (= GCP Project ID) + the **Location** custom property, with **Store Path** unused (defaulted to `n/a`). Adding Discovery in v1.2 forced this model to change. Here's why. + +The Keyfactor `IDiscoveryJobExtension` contract emits a plain `List` of discovered locations - there is no hook to set per-candidate Client Machine values. When an operator approves a discovered candidate (in the per-candidate edit dialog with `Create Certificate Store If Missing` checked), Keyfactor Command creates the new store with: + +- Store Path = the discovered location string (e.g. `projects/edgecerts/locations/global`) +- Client Machine = whatever Command auto-populated on the discovery job (typically the orchestrator hostname) - one value shared across every candidate, *not* something the operator can set per-candidate +- Custom properties = their store-type defaults + +Under the v1.1 model that meant every Discovery-approved store ended up with the *same* Client Machine across every project in the organization, which is wrong: each store needs its project ID to make GCP API calls. The first time inventory ran against a Discovery-approved store, that's exactly what produced an `HTTP 403 CONSUMER_INVALID` error against `projects//locations/global` - GCP correctly saying "that's not a valid project ID." + +##### Alternatives considered + +| Option | Why we didn't pick it | +|---|---| +| Force the operator to manually edit Client Machine after every Discovery approval | Friction. Discovery should produce working stores without an extra editing step per candidate. | +| One discovery job per project (so each job's Client Machine = that project's ID) | Impractical: an organization with 100 projects would need 100 discovery jobs, each independently configured and scheduled. | +| Have Discovery POST stores directly via Keyfactor Command's REST API instead of the standard `SubmitDiscoveryUpdate` callback | Non-standard pattern, much larger code surface, and diverges from how every other Keyfactor orchestrator works - making this orchestrator harder to maintain alongside the rest of the Keyfactor extension catalog. | +| **Make Store Path the canonical source for both manual and Discovery flows** | Picked. The discovered storepath already encodes both project and location, so reading it directly (instead of reconstructing from Client Machine + Location) means Discovery-approved stores work with zero operator edits, and manually-created stores configure the same way. Smallest code change for the cleanest user-facing schema. | + +##### Trade-offs we accepted + +- **Client Machine is now a display label**, not load-bearing. Some other Keyfactor orchestrators use Client Machine as a literal target host; for GCP that does not fit, because the orchestrator talks to a single GCP API endpoint regardless of which project a store targets - there is no per-store host to put there. The recommended value (GCP Organization ID) at least groups GCP stores together usefully in Command's UI. +- **The Location custom property is deprecated, not removed**. Keeping it in the manifest with `Required: false` preserves v1.1 stores' UI rendering during the transition. The fallback path in `JobBase.ResolveGcpResourcePath` reads it for v1.1-shaped stores (Store Path blank or `n/a`) and emits a `LogWarning` each time naming the migration step. Removal is scheduled for v2.0. +- **The Service Account Key File Path custom property is deprecated, not removed**, for the same backwards-compatibility reason. Authentication consolidates around Application Default Credentials, the GCP-recommended pattern, which works uniformly across all four job types - the deprecated property only ever worked for Inventory/Management because Discovery's UI doesn't expose store-type custom properties. Removal is scheduled for v2.0. + +#### Vendor docs + +- [Google Cloud Certificate Manager](https://cloud.google.com/certificate-manager/docs) +- [Cloud Resource Manager v3 - projects.search](https://cloud.google.com/resource-manager/reference/rest/v3/projects/search) +- [Application Default Credentials](https://cloud.google.com/docs/authentication/application-default-credentials) #### Supported Operations -| Operation | Is Supported | -|--------------|------------------------------------------------------------------------------------------------------------------------| -| Add | ✅ Checked | -| Remove | ✅ Checked | -| Discovery | ✅ Checked | +| Operation | Is Supported | +|--------------|--------------| +| Add | ✅ Checked | +| Remove | ✅ Checked | +| Discovery | ✅ Checked | | Reenrollment | 🔲 Unchecked | -| Create | ✅ Checked | +| Create | 🔲 Unchecked | #### Store Type Creation ##### Using kfutil: `kfutil` is a custom CLI for the Keyfactor Command API and can be used to create certificate store types. For more information on [kfutil](https://github.com/Keyfactor/kfutil) check out the [docs](https://github.com/Keyfactor/kfutil?tab=readme-ov-file#quickstart) +

Click to expand GcpCertMgr kfutil details ##### Using online definition from GitHub: @@ -109,10 +338,10 @@ For more information on [kfutil](https://github.com/Keyfactor/kfutil) check out ```
- #### Manual Creation Below are instructions on how to create the GcpCertMgr store type manually in the Keyfactor Command Portal +
Click to expand manual GcpCertMgr details Create a store type called `GcpCertMgr` with the attributes in the tables below: @@ -123,11 +352,11 @@ the Keyfactor Command Portal | Name | GCP Certificate Manager | Display name for the store type (may be customized) | | Short Name | GcpCertMgr | Short display name for the store type | | Capability | GcpCertMgr | Store type name orchestrator will register with. Check the box to allow entry of value | - | Supports Add | ✅ Checked | Check the box. Indicates that the Store Type supports Management Add | - | Supports Remove | ✅ Checked | Check the box. Indicates that the Store Type supports Management Remove | - | Supports Discovery | ✅ Checked | Check the box. Indicates that the Store Type supports Discovery | - | Supports Reenrollment | 🔲 Unchecked | Indicates that the Store Type supports Reenrollment | - | Supports Create | ✅ Checked | Check the box. Indicates that the Store Type supports store creation | + | Supports Add | ✅ Checked | Indicates that the Store Type supports Management Add | + | Supports Remove | ✅ Checked | Indicates that the Store Type supports Management Remove | + | Supports Discovery | ✅ Checked | Indicates that the Store Type supports Discovery | + | Supports Reenrollment | 🔲 Unchecked | Indicates that the Store Type supports Reenrollment | + | Supports Create | 🔲 Unchecked | Indicates that the Store Type supports store creation | | Needs Server | 🔲 Unchecked | Determines if a target server name is required when creating store | | Blueprint Allowed | 🔲 Unchecked | Determines if store type may be included in an Orchestrator blueprint | | Uses PowerShell | 🔲 Unchecked | Determines if underlying implementation is PowerShell | @@ -136,18 +365,18 @@ the Keyfactor Command Portal The Basic tab should look like this: - ![GcpCertMgr Basic Tab](docsource/images/GcpCertMgr-basic-store-type-dialog.png) + ![GcpCertMgr Basic Tab](docsource/images/GcpCertMgr-basic-store-type-dialog.svg) ##### Advanced Tab | Attribute | Value | Description | | --------- | ----- | ----- | | Supports Custom Alias | Required | Determines if an individual entry within a store can have a custom Alias. | - | Private Key Handling | Required | This determines if Keyfactor can send the private key associated with a certificate to the store. Required because IIS certificates without private keys would be invalid. | + | Private Key Handling | Required | This determines if Keyfactor can send the private key associated with a certificate to the store. | | PFX Password Style | Default | 'Default' - PFX password is randomly generated, 'Custom' - PFX password may be specified when the enrollment job is created (Requires the Allow Custom Password application setting to be enabled.) | The Advanced tab should look like this: - ![GcpCertMgr Advanced Tab](docsource/images/GcpCertMgr-advanced-store-type-dialog.png) + ![GcpCertMgr Advanced Tab](docsource/images/GcpCertMgr-advanced-store-type-dialog.svg) > For Keyfactor **Command versions 24.4 and later**, a Certificate Format dropdown is available with PFX and PEM options. Ensure that **PFX** is selected, as this determines the format of new and renewed certificates sent to the Orchestrator during a Management job. Currently, all Keyfactor-supported Orchestrator extensions support only PFX. @@ -156,30 +385,38 @@ the Keyfactor Command Portal | Name | Display Name | Description | Type | Default Value/Options | Required | | ---- | ------------ | ---- | --------------------- | -------- | ----------- | - | Location | Location | The GCP region used for this Certificate Manager instance. **global** is the default but could be another region based on the project. | String | global | ✅ Checked | - | ServiceAccountKey | Service Account Key File Path | The file name of the Google Cloud Service Account Key File installed in the same folder as the orchestrator extension. Empty if the orchestrator server resides in GCP and you are not using a service account key. | String | | 🔲 Unchecked | + | Location | Location (deprecated) | **Deprecated in v1.2.** The GCP location is parsed from Store Path. Leave blank for new stores. v1.1-shape stores (where Store Path is blank or `n/a`) still read this value as a fallback; expect a deprecation warning in the orchestrator log when that path is used. | String | | 🔲 Unchecked | + | ServiceAccountKey | Service Account Key File Path (deprecated) | **Deprecated in v1.2.** Leave blank. Authenticate via Application Default Credentials instead (set `GOOGLE_APPLICATION_CREDENTIALS` as a machine-level environment variable on the orchestrator host pointing at the JSON key, or run on a GCE VM / GKE pod with workload identity). The Discovery job has no way to surface this custom property in Keyfactor Command's discovery-job UI, so ADC is the only mechanism that works uniformly across all four job types. v1.1 stores that have this populated continue to work via a deprecation-logged fallback; the field is scheduled for removal in v2.0. | String | | 🔲 Unchecked | The Custom Fields tab should look like this: - ![GcpCertMgr Custom Fields Tab](docsource/images/GcpCertMgr-custom-fields-store-type-dialog.png) + ![GcpCertMgr Custom Fields Tab](docsource/images/GcpCertMgr-custom-fields-store-type-dialog.svg) + + ###### Location (deprecated) + **Deprecated in v1.2.** The GCP location is parsed from Store Path. Leave blank for new stores. v1.1-shape stores (where Store Path is blank or `n/a`) still read this value as a fallback; expect a deprecation warning in the orchestrator log when that path is used. + ![GcpCertMgr Custom Field - Location](docsource/images/GcpCertMgr-custom-field-Location-dialog.svg) - ###### Location - The GCP region used for this Certificate Manager instance. **global** is the default but could be another region based on the project. - ![GcpCertMgr Custom Field - Location](docsource/images/GcpCertMgr-custom-field-Location-dialog.png) - ![GcpCertMgr Custom Field - Location](docsource/images/GcpCertMgr-custom-field-Location-validation-options-dialog.png) + ###### Service Account Key File Path (deprecated) + **Deprecated in v1.2.** Leave blank. Authenticate via Application Default Credentials instead (set `GOOGLE_APPLICATION_CREDENTIALS` as a machine-level environment variable on the orchestrator host pointing at the JSON key, or run on a GCE VM / GKE pod with workload identity). The Discovery job has no way to surface this custom property in Keyfactor Command's discovery-job UI, so ADC is the only mechanism that works uniformly across all four job types. v1.1 stores that have this populated continue to work via a deprecation-logged fallback; the field is scheduled for removal in v2.0. + ![GcpCertMgr Custom Field - ServiceAccountKey](docsource/images/GcpCertMgr-custom-field-ServiceAccountKey-dialog.svg) - ###### Service Account Key File Path - The file name of the Google Cloud Service Account Key File installed in the same folder as the orchestrator extension. Empty if the orchestrator server resides in GCP and you are not using a service account key. + ##### Entry Parameters Tab - ![GcpCertMgr Custom Field - ServiceAccountKey](docsource/images/GcpCertMgr-custom-field-ServiceAccountKey-dialog.png) - ![GcpCertMgr Custom Field - ServiceAccountKey](docsource/images/GcpCertMgr-custom-field-ServiceAccountKey-validation-options-dialog.png) + | Name | Display Name | Description | Type | Default Value | Entry has a private key | Adding an entry | Removing an entry | Reenrolling an entry | + | ---- | ------------ | ---- | ------------- | ----------------------- | ---------------- | ----------------- | ------------------- | ----------- | + | Scope | Certificate Scope | GCP Certificate Manager `scope` for this certificate entry. Allowed: `DEFAULT` (global external Application Load Balancers), `ALL_REGIONS` (cross-region internal Application Load Balancers), `EDGE_CACHE` (Media CDN), `CLIENT_AUTH` (mTLS trust configs / authorized client server certs). **Immutable in GCP** - once a certificate is created with a given scope, GCP refuses to change it. Inventory persists the existing scope back from GCP so renewals carry it forward automatically. A single store can hold certs at different scopes (the field is per-entry, not store-wide). | MultipleChoice | DEFAULT | 🔲 Unchecked | 🔲 Unchecked | 🔲 Unchecked | 🔲 Unchecked | + The Entry Parameters tab should look like this: + ![GcpCertMgr Entry Parameters Tab](docsource/images/GcpCertMgr-entry-parameters-store-type-dialog.svg) + ##### Certificate Scope + GCP Certificate Manager `scope` for this certificate entry. Allowed: `DEFAULT` (global external Application Load Balancers), `ALL_REGIONS` (cross-region internal Application Load Balancers), `EDGE_CACHE` (Media CDN), `CLIENT_AUTH` (mTLS trust configs / authorized client server certs). **Immutable in GCP** - once a certificate is created with a given scope, GCP refuses to change it. Inventory persists the existing scope back from GCP so renewals carry it forward automatically. A single store can hold certs at different scopes (the field is per-entry, not store-wide). + ![GcpCertMgr Entry Parameter - Scope](docsource/images/GcpCertMgr-entry-parameters-store-type-dialog-Scope.svg)
@@ -188,14 +425,15 @@ the Keyfactor Command Portal 1. **Download the latest Google Cloud Provider Certificate Manager Universal Orchestrator extension from GitHub.** - Navigate to the [Google Cloud Provider Certificate Manager Universal Orchestrator extension GitHub version page](https://github.com/Keyfactor/gcp-certmanager-orchestrator/releases/latest). Refer to the compatibility matrix below to determine the asset should be downloaded. Then, click the corresponding asset to download the zip archive. + Navigate to the [Google Cloud Provider Certificate Manager Universal Orchestrator extension GitHub version page](https://github.com/Keyfactor/Google Cloud Provider Certificate Manager/releases/latest). Refer to the compatibility matrix below to determine which asset should be downloaded. Then, click the corresponding asset to download the zip archive. - | Universal Orchestrator Version | Latest .NET version installed on the Universal Orchestrator server | `rollForward` condition in `Orchestrator.runtimeconfig.json` | `gcp-certmanager-orchestrator` .NET version to download | + | Universal Orchestrator Version | Latest .NET version installed on the Universal Orchestrator server | `rollForward` condition in `Orchestrator.runtimeconfig.json` | `Google Cloud Provider Certificate Manager` .NET version to download | | --------- | ----------- | ----------- | ----------- | | Older than `11.0.0` | | | `net6.0` | | Between `11.0.0` and `11.5.1` (inclusive) | `net6.0` | | `net6.0` | - | Between `11.0.0` and `11.5.1` (inclusive) | `net8.0` | `Disable` | `net6.0` || Between `11.0.0` and `11.5.1` (inclusive) | `net8.0` | `LatestMajor` | `net8.0` | - | `11.6` _and_ newer | `net8.0` | | `net8.0` | + | Between `11.0.0` and `11.5.1` (inclusive) | `net8.0` | `Disable` | `net6.0` | + | Between `11.0.0` and `11.5.1` (inclusive) | `net8.0` | `LatestMajor` | `net8.0` | + | `11.6` _and_ newer | `net8.0` | | `net8.0` | Unzip the archive containing extension assemblies to a known location. @@ -208,25 +446,19 @@ the Keyfactor Command Portal 3. **Create a new directory for the Google Cloud Provider Certificate Manager Universal Orchestrator extension inside the extensions directory.** - Create a new directory called `gcp-certmanager-orchestrator`. + Create a new directory called `Google Cloud Provider Certificate Manager`. > The directory name does not need to match any names used elsewhere; it just has to be unique within the extensions directory. -4. **Copy the contents of the downloaded and unzipped assemblies from __step 2__ to the `gcp-certmanager-orchestrator` directory.** +4. **Copy the contents of the downloaded and unzipped assemblies from __step 2__ to the `Google Cloud Provider Certificate Manager` directory.** 5. **Restart the Universal Orchestrator service.** Refer to [Starting/Restarting the Universal Orchestrator service](https://software.keyfactor.com/Core-OnPrem/Current/Content/InstallingAgents/NetCoreOrchestrator/StarttheService.htm). - - > The above installation steps can be supplemented by the [official Command documentation](https://software.keyfactor.com/Core-OnPrem/Current/Content/InstallingAgents/NetCoreOrchestrator/CustomExtensions.htm?Highlight=extensions). - - ## Defining Certificate Stores - - ### Store Creation #### Manually with the Command UI @@ -241,20 +473,18 @@ the Keyfactor Command Portal Click the Add button to add a new Certificate Store. Use the table below to populate the **Attributes** in the **Add** form. - | Attribute | Description | - | --------- |---------------------------------------------------------| + | Attribute | Description | + | --------- | ----------- | | Category | Select "GCP Certificate Manager" or the customized certificate store name from the previous step. | | Container | Optional container to associate certificate store with. | - | Client Machine | GCP Project ID for your account. | - | Store Path | This is not used and should be defaulted to n/a per the certificate store type set up. | + | Client Machine | Display label for grouping certificate stores in Keyfactor Command. Recommended value is the GCP Organization ID (e.g. `1005564431893`); the orchestrator does not parse a project ID out of this field. The actual GCP project + location are read from Store Path. | + | Store Path | Canonical GCP resource path in the form `projects/{projectId}/locations/{location}` (e.g. `projects/edgecerts/locations/global`). This is the single source of truth for which Certificate Manager instance the store targets. For Discovery-approved stores Keyfactor Command auto-fills this from the discovered candidate; for manually-created stores the operator types it directly. | | Orchestrator | Select an approved orchestrator capable of managing `GcpCertMgr` certificates. Specifically, one with the `GcpCertMgr` capability. | - | Location | The GCP region used for this Certificate Manager instance. **global** is the default but could be another region based on the project. | - | ServiceAccountKey | The file name of the Google Cloud Service Account Key File installed in the same folder as the orchestrator extension. Empty if the orchestrator server resides in GCP and you are not using a service account key. | + | Location | **Deprecated in v1.2.** The GCP location is parsed from Store Path. Leave blank for new stores. v1.1-shape stores (where Store Path is blank or `n/a`) still read this value as a fallback; expect a deprecation warning in the orchestrator log when that path is used. | + | ServiceAccountKey | **Deprecated in v1.2.** Leave blank. Authenticate via Application Default Credentials instead (set `GOOGLE_APPLICATION_CREDENTIALS` as a machine-level environment variable on the orchestrator host pointing at the JSON key, or run on a GCE VM / GKE pod with workload identity). The Discovery job has no way to surface this custom property in Keyfactor Command's discovery-job UI, so ADC is the only mechanism that works uniformly across all four job types. v1.1 stores that have this populated continue to work via a deprecation-logged fallback; the field is scheduled for removal in v2.0. | - - #### Using kfutil CLI
Click to expand details @@ -272,11 +502,11 @@ the Keyfactor Command Portal | --------- | ----------- | | Category | Select "GCP Certificate Manager" or the customized certificate store name from the previous step. | | Container | Optional container to associate certificate store with. | - | Client Machine | GCP Project ID for your account. | - | Store Path | This is not used and should be defaulted to n/a per the certificate store type set up. | + | Client Machine | Display label for grouping certificate stores in Keyfactor Command. Recommended value is the GCP Organization ID (e.g. `1005564431893`); the orchestrator does not parse a project ID out of this field. The actual GCP project + location are read from Store Path. | + | Store Path | Canonical GCP resource path in the form `projects/{projectId}/locations/{location}` (e.g. `projects/edgecerts/locations/global`). This is the single source of truth for which Certificate Manager instance the store targets. For Discovery-approved stores Keyfactor Command auto-fills this from the discovered candidate; for manually-created stores the operator types it directly. | | Orchestrator | Select an approved orchestrator capable of managing `GcpCertMgr` certificates. Specifically, one with the `GcpCertMgr` capability. | - | Properties.Location | The GCP region used for this Certificate Manager instance. **global** is the default but could be another region based on the project. | - | Properties.ServiceAccountKey | The file name of the Google Cloud Service Account Key File installed in the same folder as the orchestrator extension. Empty if the orchestrator server resides in GCP and you are not using a service account key. | + | Properties.Location | **Deprecated in v1.2.** The GCP location is parsed from Store Path. Leave blank for new stores. v1.1-shape stores (where Store Path is blank or `n/a`) still read this value as a fallback; expect a deprecation warning in the orchestrator log when that path is used. | + | Properties.ServiceAccountKey | **Deprecated in v1.2.** Leave blank. Authenticate via Application Default Credentials instead (set `GOOGLE_APPLICATION_CREDENTIALS` as a machine-level environment variable on the orchestrator host pointing at the JSON key, or run on a GCE VM / GKE pod with workload identity). The Discovery job has no way to surface this custom property in Keyfactor Command's discovery-job UI, so ADC is the only mechanism that works uniformly across all four job types. v1.1 stores that have this populated continue to work via a deprecation-logged fallback; the field is scheduled for removal in v2.0. | 3. **Import the CSV file to create the certificate stores** @@ -286,13 +516,62 @@ the Keyfactor Command Portal
+> The content in this section can be supplemented by the [official Command documentation](https://software.keyfactor.com/Core-OnPrem/Current/Content/ReferenceGuide/Certificate%20Stores.htm?Highlight=certificate%20store). -> The content in this section can be supplemented by the [official Command documentation](https://software.keyfactor.com/Core-OnPrem/Current/Content/ReferenceGuide/Certificate%20Stores.htm?Highlight=certificate%20store). +## GCP setup prerequisites + +Before configuring the orchestrator, make sure your Google Cloud project is ready. Read the official [Google Certificate Manager](https://cloud.google.com/certificate-manager/docs) documentation for product background. The steps below are intentionally text-only; Google's Cloud Console UI changes regularly and the underlying APIs and `gcloud` commands are the stable interface. + +### 1. Enable the required Google Cloud APIs + +In the project that will host the orchestrator's service account ("the SA project"), enable both: + +- **Cloud Resource Manager API** - lets the Discovery job enumerate projects via `projects.search`. Required even if you only use Inventory/Management today, because the API enablement check runs against the SA project regardless of what target project the call reads. +- **Certificate Manager API** - read/write access to certificate resources. This must additionally be enabled in **every project** you intend to inventory or manage certs in. + +`gcloud` (one-shot for both APIs in the SA project): + +``` +gcloud services enable cloudresourcemanager.googleapis.com certificatemanager.googleapis.com --project= +``` + +### 2. Create a service account and grant organization-level roles + +Service account credentials are *identity*, not authorization - the IAM bindings determine what the SA can see and do. Bind these roles **at the organization** so the SA inherits visibility into every folder and project: + +| Role | Why | +|---|---| +| `roles/browser` | So `projects.search` returns projects nested in folders, not just top-level projects | +| `roles/certificatemanager.viewer` | Inventory: list certificates in each store | +| `roles/certificatemanager.editor` | Management/Add and Management/Remove | + +``` +gcloud iam service-accounts create kf-orchestrator \ + --project= \ + --display-name="Keyfactor Universal Orchestrator" + +ORG= +SA=kf-orchestrator@.iam.gserviceaccount.com + +gcloud organizations add-iam-policy-binding $ORG --member="serviceAccount:$SA" --role="roles/browser" +gcloud organizations add-iam-policy-binding $ORG --member="serviceAccount:$SA" --role="roles/certificatemanager.viewer" +gcloud organizations add-iam-policy-binding $ORG --member="serviceAccount:$SA" --role="roles/certificatemanager.editor" +``` + +### 3. Provide credentials to the orchestrator host (Application Default Credentials) + +The orchestrator authenticates exclusively via [Application Default Credentials](https://cloud.google.com/docs/authentication/application-default-credentials) (ADC). There are two supported deployment modes: +**Orchestrator runs inside GCP (recommended)** - on a GCE VM or GKE pod with the service account attached via workload identity. ADC discovers the service account from the metadata server automatically. No further configuration on the host. +**Orchestrator runs outside GCP** - on a Windows host, on-prem Linux, etc.: +1. Create a JSON key for the service account: `gcloud iam service-accounts keys create kf-orchestrator.json --iam-account=$SA`. Google never re-displays this key, so save it somewhere safe. +2. Copy the JSON key to a secured location on the orchestrator host. Lock down filesystem permissions so only the account that runs the Keyfactor Orchestrator service can read it. +3. Set the `GOOGLE_APPLICATION_CREDENTIALS` machine-level environment variable to the absolute path of the JSON key. Restart the Keyfactor Orchestrator service so it picks up the variable. +> **Note on the deprecated `Service Account Key File Path` store property.** Earlier versions of the orchestrator accepted a JSON filename in a per-store custom property and read the file from the orchestrator extension directory. That mechanism is deprecated in v1.2 because the Discovery job has no way to surface custom store properties in Keyfactor Command's discovery-job UI - so file-based auth can't be configured uniformly across all four job types. Existing v1.1 stores with the property populated continue to work, but every job run logs a deprecation warning. The field is scheduled for removal in v2.0. ## License @@ -300,4 +579,4 @@ Apache License 2.0, see [LICENSE](LICENSE). ## Related Integrations -See all [Keyfactor Universal Orchestrator extensions](https://github.com/orgs/Keyfactor/repositories?q=orchestrator). \ No newline at end of file +See all [Keyfactor Universal Orchestrator extensions](https://github.com/orgs/Keyfactor/repositories?q=orchestrator). diff --git a/docsource/content.md b/docsource/content.md index b1e78e5..920542b 100644 --- a/docsource/content.md +++ b/docsource/content.md @@ -2,20 +2,71 @@ The GCP Certificate Manager Orchestrator Extension remotely manages certificates on the Google Cloud Platform Certificate Manager Product. -This orchestrator extension implements three job types - Inventory, Management Add, and Management Remove. Below are the steps necessary to configure this Orchestrator Extension. It supports adding certificates with private keys only. The GCP Certificate Manager Orchestrator Extension supports the replacement of unbound certificates as well as certificates bound to existing map entries, but it does **not** support specifying map entry bindings when adding new certificates. +This orchestrator extension implements four job types - Inventory, Management Add, Management Remove, and Discovery. It supports adding certificates with private keys only. The orchestrator supports the replacement of unbound certificates as well as certificates bound to existing map entries, but it does **not** support specifying map entry bindings when adding new certificates. +### Configuration model -## Requirements +Every `GcpCertMgr` store identifies its target Certificate Manager instance through the canonical GCP resource path in **Store Path**: -**Google Cloud Configuration** +``` +projects/{projectId}/locations/{location} +``` -1. Read up on [Google Certificate Manager](https://cloud.google.com/certificate-manager/docs) and how it works. +This applies equally to manually-created stores and Discovery-approved stores. The `Location` custom property is deprecated as of v1.2 and used only as a v1.1 backwards-compatibility fallback. **Client Machine** is a display label for grouping in Command's UI - the recommended value is the GCP Organization ID. -2. Either a Google Service Account is needed with the following permissions (Note: Workload Identity Management Should be used but at the time of the writing it was not available in the .net library yet), or the virtual machine running the Keyfactor Orchestrator Service must reside within Google Cloud. -![](docsource/images/ServiceAccountSettings.gif) +The Discovery job enumerates every GCP project that the orchestrator's service account can see and proposes one candidate store per (project, location) pair, with Store Path pre-populated in canonical form. The actual scope of discovery is bounded by IAM - grant the service account the appropriate role at the organization root and Discovery will return everything underneath. See the GCP Certificate Manager store-type page for full operator-facing details. -3. The following Api Access is needed: -![](docsource/images/ApiAccessNeeded.gif) -4. If authenticating via service account, download the Json Credential file as shown below: -![](docsource/images/GoogleKeyJsonDownload.gif) \ No newline at end of file +## GCP setup prerequisites + +Before configuring the orchestrator, make sure your Google Cloud project is ready. Read the official [Google Certificate Manager](https://cloud.google.com/certificate-manager/docs) documentation for product background. The steps below are intentionally text-only; Google's Cloud Console UI changes regularly and the underlying APIs and `gcloud` commands are the stable interface. + +### 1. Enable the required Google Cloud APIs + +In the project that will host the orchestrator's service account ("the SA project"), enable both: + +- **Cloud Resource Manager API** - lets the Discovery job enumerate projects via `projects.search`. Required even if you only use Inventory/Management today, because the API enablement check runs against the SA project regardless of what target project the call reads. +- **Certificate Manager API** - read/write access to certificate resources. This must additionally be enabled in **every project** you intend to inventory or manage certs in. + +`gcloud` (one-shot for both APIs in the SA project): + +``` +gcloud services enable cloudresourcemanager.googleapis.com certificatemanager.googleapis.com --project= +``` + +### 2. Create a service account and grant organization-level roles + +Service account credentials are *identity*, not authorization - the IAM bindings determine what the SA can see and do. Bind these roles **at the organization** so the SA inherits visibility into every folder and project: + +| Role | Why | +|---|---| +| `roles/browser` | So `projects.search` returns projects nested in folders, not just top-level projects | +| `roles/certificatemanager.viewer` | Inventory: list certificates in each store | +| `roles/certificatemanager.editor` | Management/Add and Management/Remove | + +``` +gcloud iam service-accounts create kf-orchestrator \ + --project= \ + --display-name="Keyfactor Universal Orchestrator" + +ORG= +SA=kf-orchestrator@.iam.gserviceaccount.com + +gcloud organizations add-iam-policy-binding $ORG --member="serviceAccount:$SA" --role="roles/browser" +gcloud organizations add-iam-policy-binding $ORG --member="serviceAccount:$SA" --role="roles/certificatemanager.viewer" +gcloud organizations add-iam-policy-binding $ORG --member="serviceAccount:$SA" --role="roles/certificatemanager.editor" +``` + +### 3. Provide credentials to the orchestrator host (Application Default Credentials) + +The orchestrator authenticates exclusively via [Application Default Credentials](https://cloud.google.com/docs/authentication/application-default-credentials) (ADC). There are two supported deployment modes: + +**Orchestrator runs inside GCP (recommended)** - on a GCE VM or GKE pod with the service account attached via workload identity. ADC discovers the service account from the metadata server automatically. No further configuration on the host. + +**Orchestrator runs outside GCP** - on a Windows host, on-prem Linux, etc.: + +1. Create a JSON key for the service account: `gcloud iam service-accounts keys create kf-orchestrator.json --iam-account=$SA`. Google never re-displays this key, so save it somewhere safe. +2. Copy the JSON key to a secured location on the orchestrator host. Lock down filesystem permissions so only the account that runs the Keyfactor Orchestrator service can read it. +3. Set the `GOOGLE_APPLICATION_CREDENTIALS` machine-level environment variable to the absolute path of the JSON key. Restart the Keyfactor Orchestrator service so it picks up the variable. + +> **Note on the deprecated `Service Account Key File Path` store property.** Earlier versions of the orchestrator accepted a JSON filename in a per-store custom property and read the file from the orchestrator extension directory. That mechanism is deprecated in v1.2 because the Discovery job has no way to surface custom store properties in Keyfactor Command's discovery-job UI - so file-based auth can't be configured uniformly across all four job types. Existing v1.1 stores with the property populated continue to work, but every job run logs a deprecation warning. The field is scheduled for removal in v2.0. diff --git a/docsource/gcpcertmgr.md b/docsource/gcpcertmgr.md index ed37e8e..e287ddf 100644 --- a/docsource/gcpcertmgr.md +++ b/docsource/gcpcertmgr.md @@ -1 +1,240 @@ ## Overview + +The `GcpCertMgr` store type represents a single (Project, Location) pair within Google Cloud Certificate Manager. The orchestrator manages self-managed certificates inside that container - listing them for inventory, uploading new PFX certificates, and deleting existing certificates by alias. + +### Configuration model (v1.2+) + +Every `GcpCertMgr` store - whether Discovery-approved or manually created - identifies its target Certificate Manager instance through the **Store Path** field: + +``` +projects/{projectId}/locations/{location} +``` + +That single value carries both the GCP project and the location (region or `global`). Inventory and Management read it directly; **Client Machine** is a display label for grouping in Command's UI and is not parsed by the orchestrator. + +#### Field semantics + +| Field | What it carries | Read by | +|---|---|---| +| **Store Path** | Canonical GCP resource path: `projects/{projectId}/locations/{location}`. The `{location}` segment is the GCP region (or `global`) the store targets - this is the only place the orchestrator actually reads the location from for new stores. | Inventory, Management, Discovery (emit) | +| **Client Machine** | Display label only. Recommended: GCP Organization ID (e.g. `1005564431893`). Not parsed. | UI grouping in Command | +| **Service Account Key File Path** (custom, *deprecated*) | v1.1 shape only. Leave blank for new stores - authentication uses Application Default Credentials. | Credential loader fallback; emits a deprecation warning when read | +| **Location** (custom, *deprecated*) | v1.1 shape only. New stores leave it blank. Used as a fallback when Store Path is empty or `n/a`. | v1.1 fallback path; emits a deprecation warning when read | +| **Certificate Scope** (*entry parameter*, not a store property) | GCP `scope` for an individual certificate entry. One of `DEFAULT`, `ALL_REGIONS`, `EDGE_CACHE`, `CLIENT_AUTH`. Set per-cert at Add time; Inventory persists the existing value back from GCP so renewals carry it forward. Defined under the store type's EntryParameters, not Properties. See "Certificate scope" below. | Management/Add (read from `JobProperties`), Inventory (write to per-entry `Parameters`) | + +#### Location semantics: where the GCP region lives + +GCP region names (`global`, `us-central1`, `europe-west1`, ...) appear in three distinct places across the orchestrator. They look related but they are **not interchangeable**, and only one of them is load-bearing for new stores. Operators who only skim the field semantics table often miss this and end up confused about which Location field to set where. + +| # | Where it appears | What it is | What reads it | +|---|---|---|---| +| 1 | **The `{location}` segment of `Store Path`** (e.g. the `global` in `projects/edgecerts/locations/global`) | The actual GCP region the store targets. Source of truth. | Inventory and Management both call `JobBase.ResolveGcpResourcePath` which returns Store Path verbatim when it matches the canonical form. The location segment is parsed back out (string split) when the Inventory job needs to populate the `Location` parameter on each returned certificate. | +| 2 | **The `Location` custom store property** | A v1.1-shape field. New stores leave it blank. | Only the v1.1 fallback path inside `JobBase.ResolveGcpResourcePath` - it builds `projects/{ClientMachine}/locations/{Location}` when Store Path is blank or `n/a`, and emits a `LogWarning` each time naming the migration step. Removal scheduled for v2.0. | +| 3 | **The Discovery job's "Directories to search" form field** | Operator INPUT to the discovery job. A comma-separated list of regions to enumerate, e.g. `global,us-central1,europe-west1`. | `Discovery.ResolveLocations` parses the list. Discovery then emits one candidate store path per `(project × location)` combination, where the location segment of each emitted Store Path comes from this list. The list itself does **not** propagate to the resulting stores - it has no afterlife once Discovery has emitted candidates. | + +##### How the three relate + +For a **v1.2 store created via Discovery**: + +1. Operator types `global,us-central1` into the discovery job's "Directories to search" - this is place #3. +2. Discovery emits `projects/edgecerts/locations/global` and `projects/edgecerts/locations/us-central1`. Each emitted string is consumed as place #1 (Store Path) when the candidate is approved. +3. The auto-created store has Store Path populated; the **Location custom property is blank** and unused. Place #2 has no role. +4. Inventory reads Store Path (place #1) for the GCP API path, and parses the location segment back out for the `Location` parameter on each cert item. +5. Discovery's "Directories to search" value (place #3) is gone - it never landed on the store. + +For a **v1.2 store created manually**: + +- Type the full canonical path into Store Path (place #1) - e.g. `projects/edgecerts/locations/global`. +- Leave the Location custom property (place #2) blank. + +For a **v1.1 store mid-migration**: + +- Store Path is blank or `n/a`; the orchestrator falls back to building the GCP path from `Client Machine` + `Location` (place #2). A deprecation warning is logged every job run. Migrate by populating Store Path (place #1) and clearing place #2. + +##### Quick reference + +- "Where do I tell the orchestrator which GCP region to use for *this store*?" → **the `{location}` segment of Store Path** (place #1). +- "Where do I tell Discovery which regions to enumerate across *the whole org*?" → **Directories to search** on the discovery job (place #3). +- "What's the Location custom field for?" → Nothing, unless you're maintaining a v1.1 store and haven't migrated yet (place #2). +- "Why does the inventory result still have a `Location` parameter on each certificate?" → That's parsed out of Store Path's location segment for downstream filtering in Command. It mirrors place #1, not place #2. + +#### Manually creating a store + +Set: + +- **Client Machine**: GCP Organization ID +- **Store Path**: `projects/{projectId}/locations/{location}` - e.g. `projects/edgecerts/locations/global` +- **Service Account Key File Path**: leave blank (deprecated; ADC is used) +- **Location**: leave blank + +Scope is set per-cert at Add time (entry parameter, not store property) - see "Certificate scope" below. A single store can hold certificates at multiple scopes. + +Authentication uses Application Default Credentials - see "Service account credentials" below. + +#### Approving a Discovery-discovered store + +After the discovery job runs, candidates appear in **Locations → Certificate Stores → Discover** (a new tab next to "Certificate Stores"). Tick the candidates you want to track, click **MANAGE**, and Command opens the per-candidate edit dialog. The relevant fields: + +| Field | Action | +|---|---| +| **Client Machine** | Pre-filled by Command (often the orchestrator hostname). Display label only - the orchestrator does not parse it. Leave as-is, or change to the GCP Organization ID for cleaner UI grouping. | +| **Store Path** | Pre-filled with the canonical GCP path Discovery emitted (e.g. `projects/edgecerts/locations/global`). **Don't edit** - this is what Inventory and Management read. | +| **Application** | Optional, free-form. | +| **Location** (custom) | Leave blank. Deprecated v1.1 field; the location is parsed from Store Path. | +| **Service Account Key File Path** (custom) | Leave blank. Deprecated v1.1 field; authentication uses Application Default Credentials. | +| **Create Certificate Store If Missing** | **Check this.** Tells Command to create a new certificate store record from this candidate. Without it, the candidate sits in the discover tab with no store backing it. | +| **Inventory Schedule** | Pick a cadence (e.g. Daily) for the inventory job to run after the store is created. | + +Click **SAVE** and the store is created. The next inventory run on its schedule will populate it with whatever certificates exist in that (project, location). + +### Discovery job configuration + +Discovery is configured against the GCP Certificate Manager store type and enumerates candidate stores across an entire GCP organization. It uses the Cloud Resource Manager v3 API (`projects.search`) to list every active project the orchestrator's service account can see, then emits one candidate store path per (project, location) combination. + +The "Schedule Discovery" dialog inherits its layout from Keyfactor Command's generic Discovery UI, which was designed for filesystem-based store types (Java keystores, PEM files). Most fields don't apply to GCP. Here is what each field is and what to do with it: + +| Field on Schedule Discovery | What to put | +|---|---| +| **Category** | `GCP Certificate Manager` - already populated when reaching this dialog from the GCP store type. | +| **Orchestrator** | Select an approved orchestrator with the `GcpCertMgr` capability. | +| **Schedule** | When discovery should run. `Immediate` runs once on save; pick a recurring schedule for periodic re-enumeration. | +| **Directories to search** | **Required.** Type `global` for the default behavior of searching only GCP's global Certificate Manager location, which is what almost every operator wants. See "Should I ever put something other than `global`?" below for the rare exceptions. | +| **Directories to ignore** | Leave blank. Filesystem-store concept; not used by GCP discovery. | +| **Extensions** | Leave blank. Filesystem-store concept; not used. | +| **File name patterns to match** | Leave blank. Filesystem-store concept; not used. | +| **Follow SymLinks** | Leave unchecked. Filesystem-store concept; not used. | +| **Include PKCS12 Files?** | Leave unchecked. Filesystem-store concept; not used. | + +> **Why are most fields irrelevant to GCP?** Command's Discovery UI is one form template shared across every store type. For filesystem-based store types like Java keystores or PEM files, fields like *Directories to ignore*, *Extensions*, *File name patterns to match*, *Follow SymLinks*, and *Include PKCS12 Files?* are useful - they let the orchestrator narrow which files on disk it should treat as candidate stores. GCP Certificate Manager isn't a filesystem; the orchestrator uses Cloud Resource Manager + Certificate Manager APIs, so these fields are not consulted. The orchestrator does not raise an error if you fill them in; it just ignores them. + +#### Should I ever put something other than `global`? + +Almost never. Concrete guidance: + +- **Just type `global` → searches the `global` GCP location only.** This is the right answer for the vast majority of GCP Certificate Manager deployments, because certificates attached to GCP's *global* external Application Load Balancer (the most common load balancer in GCP) are stored in the `global` Certificate Manager location. +- **Add specific regions** (e.g. `global,us-central1,europe-west1`) only if your organization runs **regional** external Application Load Balancers, or has data-residency requirements that pin certificates to specific regions. If you're not sure whether that describes your environment, the answer is "you don't need this" and you should just type `global`. +- **Don't list every GCP region** (`us-central1,us-east1,...`). Discovery does not probe candidates - it emits one (project × location) pair regardless of whether that combination has any certs. Listing 40 regions for a 100-project org produces 4,000 candidate stores, most empty, all cluttering Command's certificate store list. + +The format is a comma-separated list of GCP location names exactly as GCP names them. `global` is the universal location; regional names follow GCP's standard `-` form (e.g. `us-central1`, `europe-west1`, `asia-southeast1`). See the [Certificate Manager supported locations list](https://cloud.google.com/certificate-manager/docs/locations) for the canonical set. + +The candidate count is always `projects × locations`, so each region you add multiplies the size of the discovery result by the number of accessible projects. + +#### Service account credentials + +The orchestrator authenticates exclusively via Application Default Credentials. Two supported deployment modes: + +- **Inside GCP** - on a GCE VM or GKE pod with the service account attached via workload identity. ADC discovers the service account from the metadata server automatically. No host configuration needed. +- **Outside GCP** - on a Windows host or on-prem Linux. Set the `GOOGLE_APPLICATION_CREDENTIALS` machine-level environment variable to the absolute path of the service account's JSON key, then restart the Keyfactor Orchestrator service so it picks up the variable. The account that runs the orchestrator service must have read access to the JSON key file. + +The legacy `Service Account Key File Path` custom store property (a JSON filename relative to the orchestrator extension directory) is **deprecated as of v1.2** because the Discovery job has no way to surface custom store properties in Keyfactor Command's discovery-job UI - so file-based auth can't be configured uniformly across all four job types. v1.1 stores with the property populated continue to work, but every job run logs a deprecation warning. The field is scheduled for removal in v2.0; new stores should leave it blank. + +The service account needs at minimum: + +- `roles/browser` at the **organization** root - for `projects.search` to see projects nested in folders. +- `roles/certificatemanager.viewer` per project (or at the org root for inheritance) - for inventory to list certificates. +- `roles/certificatemanager.editor` - for management to add/remove certificates. + +Required APIs to enable in the **service account's home project**: + +- Cloud Resource Manager API +- Certificate Manager API (also needs to be enabled in every project you actually inventory) + +### Certificate alias rules + +GCP Certificate Manager constrains certificate resource IDs to a strict shape: + +- 1 to 63 characters +- Lowercase letters, digits, hyphens only +- Must start with a lowercase letter +- Must not end with a hyphen +- Regex: `[a-z]([-a-z0-9]*[a-z0-9])?` + +The orchestrator validates the alias against this rule **before** any API calls or PFX parsing during Management/Add. A non-conforming alias fails fast with a `[FAIL] ValidateAlias` step in the flow trace and a suggestion of a normalized alias (e.g. `Cert1` → `cert1`). Rename the certificate in Keyfactor Command to the suggested form and retry the Management/Add job. + +### Certificate scope + +GCP Certificate Manager attaches a `scope` to every certificate that determines which load balancer / service families can consume it. The orchestrator models this as a per-entry **entry parameter** (defined under `EntryParameters` on the store type), not a store property. That matches GCP's own data model - a single (project, location) container in GCP can legitimately hold certificates with different scopes. + +| Value | What it is for | +|---|---| +| `DEFAULT` | Global external Application Load Balancers - the standard GCP load balancer for internet-facing traffic. This is the GCP default and the right answer for most certs. | +| `ALL_REGIONS` | Cross-region **internal** Application Load Balancers. Use this when the consuming load balancer is regional/internal and replicated across regions. | +| `EDGE_CACHE` | Google Cloud Media CDN edge-cache certs. | +| `CLIENT_AUTH` | Certificates used by mTLS trust configs, or server certificates that are authorized for client authentication. | + +#### How operators set it + +On Management/Add (uploading a new cert into a store), Keyfactor Command renders a dropdown for "Certificate Scope" sourced from the store type's `EntryParameters` definition. Pick one of the four values; default is `DEFAULT`. The chosen value lands in `ManagementJobConfiguration.JobProperties["Scope"]` and the orchestrator passes it to GCP on the `certificates.create` call. + +On renewals / reenrollments, Keyfactor pre-fills the dropdown from the certificate's last-known inventory parameters, so the cert keeps its scope automatically across renewal cycles without operator action. + +#### Immutability + +The `scope` field is **create-only** in the GCP API. Once GCP creates a certificate with a given scope, that scope cannot be changed by any patch operation. The orchestrator's Management/Replace path uses `UpdateMask = "SelfManaged"`, so re-adding a certificate over an existing one preserves its original scope - even if the operator selects a different scope in the renewal dialog. To migrate a certificate to a different scope, delete it (Management/Remove) and re-add it with the new scope. + +#### Inventory persists scope per-cert + +The Inventory job reads the `scope` field off each `certificates.list` response and writes it into the cert's `CurrentInventoryItem.Parameters` dict. GCP omits the field from the response when the cert is at the default scope; the orchestrator normalizes null/blank to `DEFAULT` so Command's UI always shows a concrete value. After the first inventory run on a store, every cert displays its actual GCP scope in Command, and that value flows back through to Management jobs on subsequent renew/reenrollment operations. + +#### What happened before v1.2.1 + +Prior to v1.2.1 the orchestrator hard-coded `Scope = "DEFAULT"` on every certificate it created and never read `scope` back during Inventory. Customers who needed non-default scopes (typically `ALL_REGIONS` for cross-region internal ALBs) had to pre-create empty placeholder certificate resources in GCP via Terraform with `scope = "ALL_REGIONS"`, then point Keyfactor at the existing resource as a Replace target. The new entry parameter removes that workaround. + +#### How the orchestrator validates the value + +`JobBase.ResolveScope` runs as the `ResolveScope` flow step on every Management/Add. It trims and uppercases the configured value, then validates it against the set GCP accepts. An unsupported value (typo, lowercase letters that don't normalize to a valid token, anything outside the four allowed values) fails with `[FAIL] ResolveScope` and a clear message naming the four legal values. Blank or null resolves to `DEFAULT`. The dropdown UI in Command should prevent invalid values from reaching the orchestrator in the first place; `ResolveScope` is defence-in-depth for direct-API submissions. + +#### Quick reference + +- "Where do I see what scope a certificate ended up with?" → Run inventory once - it surfaces in the cert's Parameters in Command. Also `gcloud certificate-manager certificates describe --location= --project=`. +- "Can I change a certificate's scope?" → No. Delete and re-add. +- "I have one store with DEFAULT certs and I want to add an ALL_REGIONS cert" → Add the new cert with Scope = `ALL_REGIONS` from the dropdown. Existing DEFAULT certs in the same store are unaffected; the field is per-entry, not store-wide. + +### Architecture and logging + +Every job (Discovery, Inventory, Management) uses a shared `FlowLogger` to record step-by-step progress with timing. The flow summary is appended to `JobResult.FailureMessage` on **both** success and failure paths so operators reading job history can see what happened without having to pull orchestrator-side trace logs. Errors arising from the GCP SDK are unwrapped through `AggregateException` walls and reported with HTTP status + the GCP error response body, so quota errors / IAM denials / malformed certificates surface clearly in Command's UI. + +### Migrating v1.1 stores + +A v1.1-shape store has `Store Path` empty or `n/a`, `Client Machine` set to the GCP Project ID, the `Location` custom property set to the region, and possibly the `Service Account Key File Path` custom property pointing at a JSON key in the orchestrator extension directory. These continue to work in v1.2 through fallback paths, but every inventory/management run logs deprecation warnings naming the store. To migrate, edit each affected store: + +1. Set **Store Path** to `projects/{the-current-Client-Machine-value}/locations/{the-current-Location-value}`. +2. Optionally change **Client Machine** to the GCP Organization ID for cleaner UI grouping. +3. Optionally clear the **Location** field (no longer required). +4. Configure ADC on the orchestrator host (see "Service account credentials") and clear the **Service Account Key File Path** field. +5. Save. + +The deprecation warnings will stop on the next job run once the store is fully migrated. Both fallbacks will be removed in v2.0. + +### Design rationale: why Store Path is the source of truth + +In v1.1 the orchestrator built the GCP resource path from **Client Machine** (= GCP Project ID) + the **Location** custom property, with **Store Path** unused (defaulted to `n/a`). Adding Discovery in v1.2 forced this model to change. Here's why. + +The Keyfactor `IDiscoveryJobExtension` contract emits a plain `List` of discovered locations - there is no hook to set per-candidate Client Machine values. When an operator approves a discovered candidate (in the per-candidate edit dialog with `Create Certificate Store If Missing` checked), Keyfactor Command creates the new store with: + +- Store Path = the discovered location string (e.g. `projects/edgecerts/locations/global`) +- Client Machine = whatever Command auto-populated on the discovery job (typically the orchestrator hostname) - one value shared across every candidate, *not* something the operator can set per-candidate +- Custom properties = their store-type defaults + +Under the v1.1 model that meant every Discovery-approved store ended up with the *same* Client Machine across every project in the organization, which is wrong: each store needs its project ID to make GCP API calls. The first time inventory ran against a Discovery-approved store, that's exactly what produced an `HTTP 403 CONSUMER_INVALID` error against `projects//locations/global` - GCP correctly saying "that's not a valid project ID." + +#### Alternatives considered + +| Option | Why we didn't pick it | +|---|---| +| Force the operator to manually edit Client Machine after every Discovery approval | Friction. Discovery should produce working stores without an extra editing step per candidate. | +| One discovery job per project (so each job's Client Machine = that project's ID) | Impractical: an organization with 100 projects would need 100 discovery jobs, each independently configured and scheduled. | +| Have Discovery POST stores directly via Keyfactor Command's REST API instead of the standard `SubmitDiscoveryUpdate` callback | Non-standard pattern, much larger code surface, and diverges from how every other Keyfactor orchestrator works - making this orchestrator harder to maintain alongside the rest of the Keyfactor extension catalog. | +| **Make Store Path the canonical source for both manual and Discovery flows** | Picked. The discovered storepath already encodes both project and location, so reading it directly (instead of reconstructing from Client Machine + Location) means Discovery-approved stores work with zero operator edits, and manually-created stores configure the same way. Smallest code change for the cleanest user-facing schema. | + +#### Trade-offs we accepted + +- **Client Machine is now a display label**, not load-bearing. Some other Keyfactor orchestrators use Client Machine as a literal target host; for GCP that does not fit, because the orchestrator talks to a single GCP API endpoint regardless of which project a store targets - there is no per-store host to put there. The recommended value (GCP Organization ID) at least groups GCP stores together usefully in Command's UI. +- **The Location custom property is deprecated, not removed**. Keeping it in the manifest with `Required: false` preserves v1.1 stores' UI rendering during the transition. The fallback path in `JobBase.ResolveGcpResourcePath` reads it for v1.1-shaped stores (Store Path blank or `n/a`) and emits a `LogWarning` each time naming the migration step. Removal is scheduled for v2.0. +- **The Service Account Key File Path custom property is deprecated, not removed**, for the same backwards-compatibility reason. Authentication consolidates around Application Default Credentials, the GCP-recommended pattern, which works uniformly across all four job types - the deprecated property only ever worked for Inventory/Management because Discovery's UI doesn't expose store-type custom properties. Removal is scheduled for v2.0. + +### Vendor docs + +- [Google Cloud Certificate Manager](https://cloud.google.com/certificate-manager/docs) +- [Cloud Resource Manager v3 - projects.search](https://cloud.google.com/resource-manager/reference/rest/v3/projects/search) +- [Application Default Credentials](https://cloud.google.com/docs/authentication/application-default-credentials) diff --git a/docsource/images/ApiAccessNeeded.gif b/docsource/images/ApiAccessNeeded.gif deleted file mode 100644 index 8a139eb..0000000 Binary files a/docsource/images/ApiAccessNeeded.gif and /dev/null differ diff --git a/docsource/images/GcpCertMgr-advanced-store-type-dialog.svg b/docsource/images/GcpCertMgr-advanced-store-type-dialog.svg new file mode 100644 index 0000000..123a979 --- /dev/null +++ b/docsource/images/GcpCertMgr-advanced-store-type-dialog.svg @@ -0,0 +1,67 @@ + + + + + + + + + Edit Certificate Store Type + + + + Basic + Advanced + + Custom Fields + Entry Parameters + + + + + Store Path Type + + + + Freeform + + Fixed + + Multiple Choice + + + + + Other Settings + + Supports Custom Alias + + Forbidden + + Optional + + + Required + Private Key Handling + + Forbidden + + Optional + + + Required + PFX Password Style + + + Default + + Custom + \ No newline at end of file diff --git a/docsource/images/GcpCertMgr-basic-store-type-dialog.svg b/docsource/images/GcpCertMgr-basic-store-type-dialog.svg new file mode 100644 index 0000000..a69470d --- /dev/null +++ b/docsource/images/GcpCertMgr-basic-store-type-dialog.svg @@ -0,0 +1,83 @@ + + + + + + + + + Edit Certificate Store Type + + + + Basic + + Advanced + Custom Fields + Entry Parameters + + + + + Details + + Name + + GCP Certificate Manager + Short Name + + GcpCertMgr + Custom Capability + + + Custom Capability + + + + Supported Job Types + + + + Inventory + + + Add + + + Remove + + Create + + + Discovery + + ODKG + + + + General Settings + + + Needs Server + + Blueprint Allowed + + Uses PowerShell + + + + Password Settings + + + Requires Store Password + + Supports Entry Password + \ No newline at end of file diff --git a/docsource/images/GcpCertMgr-custom-field-Location-dialog.svg b/docsource/images/GcpCertMgr-custom-field-Location-dialog.svg new file mode 100644 index 0000000..a3cf7e6 --- /dev/null +++ b/docsource/images/GcpCertMgr-custom-field-Location-dialog.svg @@ -0,0 +1,49 @@ + + + + + + + + + Edit Custom Field + × + + + + Basic Information + + Validation Options + + Name + + Location + Display Name + + Location (deprecated) + Type + + String + ∨ + Default Value + + + Depends On + + + Service Account Key File Path (deprecated) + ∨ + + + CANCEL + + SAVE + \ No newline at end of file diff --git a/docsource/images/GcpCertMgr-custom-field-Location-validation-options-dialog.svg b/docsource/images/GcpCertMgr-custom-field-Location-validation-options-dialog.svg new file mode 100644 index 0000000..22f8bbd --- /dev/null +++ b/docsource/images/GcpCertMgr-custom-field-Location-validation-options-dialog.svg @@ -0,0 +1,39 @@ + + + + + + + + + Edit Custom Field + × + + + + Basic Information + Validation Options + + + Creating a certificate store + + + Optional + + Required + + Hidden + + + CANCEL + + SAVE + \ No newline at end of file diff --git a/docsource/images/GcpCertMgr-custom-field-ServiceAccountKey-dialog.svg b/docsource/images/GcpCertMgr-custom-field-ServiceAccountKey-dialog.svg new file mode 100644 index 0000000..19d5e4f --- /dev/null +++ b/docsource/images/GcpCertMgr-custom-field-ServiceAccountKey-dialog.svg @@ -0,0 +1,49 @@ + + + + + + + + + Edit Custom Field + × + + + + Basic Information + + Validation Options + + Name + + ServiceAccountKey + Display Name + + Service Account Key File Path (deprecated) + Type + + String + ∨ + Default Value + + + Depends On + + + Location (deprecated) + ∨ + + + CANCEL + + SAVE + \ No newline at end of file diff --git a/docsource/images/GcpCertMgr-custom-field-ServiceAccountKey-validation-options-dialog.svg b/docsource/images/GcpCertMgr-custom-field-ServiceAccountKey-validation-options-dialog.svg new file mode 100644 index 0000000..22f8bbd --- /dev/null +++ b/docsource/images/GcpCertMgr-custom-field-ServiceAccountKey-validation-options-dialog.svg @@ -0,0 +1,39 @@ + + + + + + + + + Edit Custom Field + × + + + + Basic Information + Validation Options + + + Creating a certificate store + + + Optional + + Required + + Hidden + + + CANCEL + + SAVE + \ No newline at end of file diff --git a/docsource/images/GcpCertMgr-custom-fields-store-type-dialog.svg b/docsource/images/GcpCertMgr-custom-fields-store-type-dialog.svg new file mode 100644 index 0000000..8bbc791 --- /dev/null +++ b/docsource/images/GcpCertMgr-custom-fields-store-type-dialog.svg @@ -0,0 +1,62 @@ + + + + + + + + + Edit Certificate Store Type + + + + Basic + Advanced + Custom Fields + + Entry Parameters + + + + + + ADD + + EDIT + + DELETE + Total: 2 + + + Display Name + Type + Default Value / Options + + + + + + + + + + + Location (deprecated) + String + + + + + + + Service Account Key File Path (dep... + String + \ No newline at end of file diff --git a/docsource/images/GcpCertMgr-entry-parameters-store-type-dialog-Scope-validation-options.svg b/docsource/images/GcpCertMgr-entry-parameters-store-type-dialog-Scope-validation-options.svg new file mode 100644 index 0000000..ddaa9ab --- /dev/null +++ b/docsource/images/GcpCertMgr-entry-parameters-store-type-dialog-Scope-validation-options.svg @@ -0,0 +1,68 @@ + + + + + + + + + Edit Entry Parameter + × + + + + Basic Information + Validation Options + + + Entry has a private key + + + Optional + + Required + + Ignore + + + + Job Types + + Adding an entry + + + Optional + + Required + + Hidden + Removing an entry + + + Optional + + Required + + Hidden + On Device Key Generation + + + Optional + + Required + + Hidden + + + CANCEL + + SAVE + \ No newline at end of file diff --git a/docsource/images/GcpCertMgr-entry-parameters-store-type-dialog-Scope.svg b/docsource/images/GcpCertMgr-entry-parameters-store-type-dialog-Scope.svg new file mode 100644 index 0000000..625957a --- /dev/null +++ b/docsource/images/GcpCertMgr-entry-parameters-store-type-dialog-Scope.svg @@ -0,0 +1,51 @@ + + + + + + + + + Edit Entry Parameter + × + + + + Basic Information + + Validation Options + + Name + + Scope + Display Name + + Certificate Scope + Type + + MultipleChoice + ∨ + Default Value + + DEFAULT + Multiple Choice Options + + DEFAULT,ALL_REGIONS,EDGE_CACHE,CLIENT_AUTH + Depends On + + + ∨ + + + CANCEL + + SAVE + \ No newline at end of file diff --git a/docsource/images/GcpCertMgr-entry-parameters-store-type-dialog.svg b/docsource/images/GcpCertMgr-entry-parameters-store-type-dialog.svg new file mode 100644 index 0000000..2a8bb8a --- /dev/null +++ b/docsource/images/GcpCertMgr-entry-parameters-store-type-dialog.svg @@ -0,0 +1,55 @@ + + + + + + + + + Edit Certificate Store Type + + + + Basic + Advanced + Custom Fields + Entry Parameters + + + + + + + ADD + + EDIT + + DELETE + Total: 1 + + + Display Name + Type + Default Value + + + + + + + + + + + Certificate Scope + MultipleChoice + DEFAULT + \ No newline at end of file diff --git a/docsource/images/GoogleKeyJsonDownload.gif b/docsource/images/GoogleKeyJsonDownload.gif deleted file mode 100644 index b617c7a..0000000 Binary files a/docsource/images/GoogleKeyJsonDownload.gif and /dev/null differ diff --git a/docsource/images/ServiceAccountSettings.gif b/docsource/images/ServiceAccountSettings.gif deleted file mode 100644 index b61d1bd..0000000 Binary files a/docsource/images/ServiceAccountSettings.gif and /dev/null differ diff --git a/integration-manifest.json b/integration-manifest.json index da19f33..3f22e87 100644 --- a/integration-manifest.json +++ b/integration-manifest.json @@ -25,10 +25,10 @@ "PowerShell": false, "PrivateKeyAllowed": "Required", "StorePathType": "", - "StorePathValue": "n/a", + "StorePathValue": "", "SupportedOperations": { "Add": true, - "Create": true, + "Create": false, "Discovery": true, "Enrollment": false, "Remove": true @@ -41,28 +41,44 @@ "Properties": [ { "Name": "Location", - "DisplayName": "Location", + "DisplayName": "Location (deprecated)", "Type": "String", "DependsOn": "", - "DefaultValue": "global", - "Required": true, + "DefaultValue": "", + "Required": false, "IsPAMEligible": false, - "Description": "The GCP region used for this Certificate Manager instance. **global** is the default but could be another region based on the project." + "Description": "**Deprecated in v1.2.** The GCP location is parsed from Store Path. Leave blank for new stores. v1.1-shape stores (where Store Path is blank or `n/a`) still read this value as a fallback; expect a deprecation warning in the orchestrator log when that path is used." }, { "Name": "ServiceAccountKey", - "DisplayName": "Service Account Key File Path", + "DisplayName": "Service Account Key File Path (deprecated)", "Type": "String", "DependsOn": "", "DefaultValue": "", "Required": false, "IsPAMEligible": false, - "Description": "The file name of the Google Cloud Service Account Key File installed in the same folder as the orchestrator extension. Empty if the orchestrator server resides in GCP and you are not using a service account key." + "Description": "**Deprecated in v1.2.** Leave blank. Authenticate via Application Default Credentials instead (set `GOOGLE_APPLICATION_CREDENTIALS` as a machine-level environment variable on the orchestrator host pointing at the JSON key, or run on a GCE VM / GKE pod with workload identity). The Discovery job has no way to surface this custom property in Keyfactor Command's discovery-job UI, so ADC is the only mechanism that works uniformly across all four job types. v1.1 stores that have this populated continue to work via a deprecation-logged fallback; the field is scheduled for removal in v2.0." } ], - "ClientMachineDescription": "GCP Project ID for your account.", - "StorePathDescription": "This is not used and should be defaulted to n/a per the certificate store type set up.", - "EntryParameters": [] + "ClientMachineDescription": "Display label for grouping certificate stores in Keyfactor Command. Recommended value is the GCP Organization ID (e.g. `1005564431893`); the orchestrator does not parse a project ID out of this field. The actual GCP project + location are read from Store Path.", + "StorePathDescription": "Canonical GCP resource path in the form `projects/{projectId}/locations/{location}` (e.g. `projects/edgecerts/locations/global`). This is the single source of truth for which Certificate Manager instance the store targets. For Discovery-approved stores Keyfactor Command auto-fills this from the discovered candidate; for manually-created stores the operator types it directly.", + "EntryParameters": [ + { + "Name": "Scope", + "DisplayName": "Certificate Scope", + "Type": "MultipleChoice", + "DependsOn": "", + "DefaultValue": "DEFAULT", + "Options": "DEFAULT,ALL_REGIONS,EDGE_CACHE,CLIENT_AUTH", + "RequiredWhen": { + "HasPrivateKey": false, + "OnAdd": false, + "OnRemove": false, + "OnReenrollment": false + }, + "Description": "GCP Certificate Manager `scope` for this certificate entry. Allowed: `DEFAULT` (global external Application Load Balancers), `ALL_REGIONS` (cross-region internal Application Load Balancers), `EDGE_CACHE` (Media CDN), `CLIENT_AUTH` (mTLS trust configs / authorized client server certs). **Immutable in GCP** - once a certificate is created with a given scope, GCP refuses to change it. Inventory persists the existing scope back from GCP so renewals carry it forward automatically. A single store can hold certs at different scopes (the field is per-entry, not store-wide)." + } + ] } ] } diff --git a/scripts/store_types/bash/curl_create_store_types.sh b/scripts/store_types/bash/curl_create_store_types.sh index 2c97865..40ca05c 100755 --- a/scripts/store_types/bash/curl_create_store_types.sh +++ b/scripts/store_types/bash/curl_create_store_types.sh @@ -1,78 +1,20 @@ -#!/usr/bin/env bash +#!/bin/bash +# Store Type creation script using curl +# Generated by Doctool -# Creates all 1 store types via the Keyfactor Command REST API using curl. -# -# Authentication (first matching method is used): -# OAuth access token: KEYFACTOR_AUTH_ACCESS_TOKEN -# OAuth client creds: KEYFACTOR_AUTH_CLIENT_ID + KEYFACTOR_AUTH_CLIENT_SECRET -# + KEYFACTOR_AUTH_TOKEN_URL -# Basic auth (AD): KEYFACTOR_USERNAME + KEYFACTOR_PASSWORD + KEYFACTOR_DOMAIN -# -# Always required: -# KEYFACTOR_HOSTNAME Command hostname (e.g. my-command.example.com) -# -# Auto-generated by doctool generate-store-type-scripts — do not edit by hand. +set -e -if [ -z "${KEYFACTOR_HOSTNAME}" ]; then - echo "ERROR: KEYFACTOR_HOSTNAME is required" - exit 1 -fi +# Configuration - set these variables before running +KEYFACTOR_HOSTNAME="${KEYFACTOR_HOSTNAME}" +KEYFACTOR_API_PATH="${KEYFACTOR_API_PATH:-KeyfactorAPI}" +KEYFACTOR_AUTH_TOKEN="${KEYFACTOR_AUTH_TOKEN}" -BASE_URL="https://${KEYFACTOR_HOSTNAME}/keyfactorapi" - -# --------------------------------------------------------------------------- -# Resolve auth -# --------------------------------------------------------------------------- -if [ -n "${KEYFACTOR_AUTH_ACCESS_TOKEN}" ]; then - BEARER_TOKEN="${KEYFACTOR_AUTH_ACCESS_TOKEN}" -elif [ -n "${KEYFACTOR_AUTH_CLIENT_ID}" ] && [ -n "${KEYFACTOR_AUTH_CLIENT_SECRET}" ] && [ -n "${KEYFACTOR_AUTH_TOKEN_URL}" ]; then - echo "Fetching OAuth token..." - BEARER_TOKEN=$(curl -s -X POST "${KEYFACTOR_AUTH_TOKEN_URL}" \ - -H "Content-Type: application/x-www-form-urlencoded" \ - --data-urlencode "grant_type=client_credentials" \ - --data-urlencode "client_id=${KEYFACTOR_AUTH_CLIENT_ID}" \ - --data-urlencode "client_secret=${KEYFACTOR_AUTH_CLIENT_SECRET}" | jq -r '.access_token') - if [ -z "${BEARER_TOKEN}" ] || [ "${BEARER_TOKEN}" = "null" ]; then - echo "ERROR: Failed to fetch OAuth token from ${KEYFACTOR_AUTH_TOKEN_URL}" - exit 1 - fi -elif [ -n "${KEYFACTOR_USERNAME}" ] && [ -n "${KEYFACTOR_PASSWORD}" ] && [ -n "${KEYFACTOR_DOMAIN}" ]; then - BEARER_TOKEN="" -else - echo "ERROR: Authentication required. Set one of:" - echo " KEYFACTOR_AUTH_ACCESS_TOKEN" - echo " KEYFACTOR_AUTH_CLIENT_ID + KEYFACTOR_AUTH_CLIENT_SECRET + KEYFACTOR_AUTH_TOKEN_URL" - echo " KEYFACTOR_USERNAME + KEYFACTOR_PASSWORD + KEYFACTOR_DOMAIN" - exit 1 -fi - -if [ -n "${BEARER_TOKEN}" ]; then - CURL_AUTH=("-H" "Authorization: Bearer ${BEARER_TOKEN}") -else - CURL_AUTH=("-u" "${KEYFACTOR_USERNAME}@${KEYFACTOR_DOMAIN}:${KEYFACTOR_PASSWORD}") -fi - -create_store_type() { - local name="$1" - local body="$2" - echo "Creating ${name} store type..." - response=$(curl -s -o /dev/null -w "%{http_code}" \ - -X POST "${BASE_URL}/certificatestoretypes" \ - -H "Content-Type: application/json" \ - -H "x-keyfactor-requested-with: APIClient" \ - "${CURL_AUTH[@]}" \ - -d "${body}") - if [ "$response" = "200" ] || [ "$response" = "201" ]; then - echo " OK (HTTP ${response})" - else - echo " FAILED (HTTP ${response})" - fi -} - -# --------------------------------------------------------------------------- -# GcpCertMgr — GCP Project ID for your account. -# --------------------------------------------------------------------------- -create_store_type "GcpCertMgr" '{ +echo "Creating store type: GcpCertMgr" +curl -s -X POST "https://${KEYFACTOR_HOSTNAME}/${KEYFACTOR_API_PATH}/CertificateStoreTypes" \ + -H "Authorization: Bearer ${KEYFACTOR_AUTH_TOKEN}" \ + -H "Content-Type: application/json" \ + -H "x-keyfactor-requested-with: APIClient" \ + -d '{ "Name": "GCP Certificate Manager", "ShortName": "GcpCertMgr", "Capability": "GcpCertMgr", @@ -82,10 +24,10 @@ create_store_type "GcpCertMgr" '{ "PowerShell": false, "PrivateKeyAllowed": "Required", "StorePathType": "", - "StorePathValue": "n/a", + "StorePathValue": "", "SupportedOperations": { "Add": true, - "Create": true, + "Create": false, "Discovery": true, "Enrollment": false, "Remove": true @@ -98,26 +40,41 @@ create_store_type "GcpCertMgr" '{ "Properties": [ { "Name": "Location", - "DisplayName": "Location", + "DisplayName": "Location (deprecated)", "Type": "String", "DependsOn": "", - "DefaultValue": "global", - "Required": true, - "IsPAMEligible": false + "DefaultValue": "", + "Required": false, + "IsPAMEligible": false, + "Description": "**Deprecated in v1.2.** The GCP location is parsed from Store Path. Leave blank for new stores. v1.1-shape stores (where Store Path is blank or `n/a`) still read this value as a fallback; expect a deprecation warning in the orchestrator log when that path is used." }, { "Name": "ServiceAccountKey", - "DisplayName": "Service Account Key File Path", + "DisplayName": "Service Account Key File Path (deprecated)", "Type": "String", "DependsOn": "", "DefaultValue": "", "Required": false, - "IsPAMEligible": false + "IsPAMEligible": false, + "Description": "**Deprecated in v1.2.** Leave blank. Authenticate via Application Default Credentials instead (set `GOOGLE_APPLICATION_CREDENTIALS` as a machine-level environment variable on the orchestrator host pointing at the JSON key, or run on a GCE VM / GKE pod with workload identity). The Discovery job has no way to surface this custom property in Keyfactor Command's discovery-job UI, so ADC is the only mechanism that works uniformly across all four job types. v1.1 stores that have this populated continue to work via a deprecation-logged fallback; the field is scheduled for removal in v2.0." } ], - "StorePathDescription": "This is not used and should be defaulted to n/a per the certificate store type set up.", - "EntryParameters": [] + "EntryParameters": [ + { + "Name": "Scope", + "DisplayName": "Certificate Scope", + "Type": "MultipleChoice", + "DependsOn": "", + "DefaultValue": "DEFAULT", + "Options": "DEFAULT,ALL_REGIONS,EDGE_CACHE,CLIENT_AUTH", + "RequiredWhen": { + "HasPrivateKey": false, + "OnAdd": false, + "OnRemove": false, + "OnReenrollment": false + }, + "Description": "GCP Certificate Manager `scope` for this certificate entry. Allowed: `DEFAULT` (global external Application Load Balancers), `ALL_REGIONS` (cross-region internal Application Load Balancers), `EDGE_CACHE` (Media CDN), `CLIENT_AUTH` (mTLS trust configs / authorized client server certs). **Immutable in GCP** - once a certificate is created with a given scope, GCP refuses to change it. Inventory persists the existing scope back from GCP so renewals carry it forward automatically. A single store can hold certs at different scopes (the field is per-entry, not store-wide)." + } + ] }' - -echo "Completed." diff --git a/scripts/store_types/bash/kfutil_create_store_types.sh b/scripts/store_types/bash/kfutil_create_store_types.sh index 06c596c..e6dced1 100755 --- a/scripts/store_types/bash/kfutil_create_store_types.sh +++ b/scripts/store_types/bash/kfutil_create_store_types.sh @@ -1,28 +1,9 @@ -#!/usr/bin/env bash +#!/bin/bash +# Store Type creation script using kfutil +# Generated by Doctool -# Creates all 1 store types using kfutil. -# kfutil reads definitions from the Keyfactor integration catalog. -# -# Auth environment variables (first matching method is used): -# OAuth access token: KEYFACTOR_AUTH_ACCESS_TOKEN -# OAuth client creds: KEYFACTOR_AUTH_CLIENT_ID + KEYFACTOR_AUTH_CLIENT_SECRET -# + KEYFACTOR_AUTH_TOKEN_URL -# Basic auth (AD): KEYFACTOR_HOSTNAME + KEYFACTOR_USERNAME + KEYFACTOR_PASSWORD -# + KEYFACTOR_DOMAIN -# -# Auto-generated by doctool generate-store-type-scripts — do not edit by hand. +set -e -if ! command -v kfutil &> /dev/null; then - echo "kfutil could not be found. Please install kfutil" - echo "See https://github.com/Keyfactor/kfutil#quickstart" - exit 1 -fi +echo "Creating store type: GcpCertMgr" +kfutil store-types create GcpCertMgr -if [ -z "$KEYFACTOR_HOSTNAME" ]; then - echo "KEYFACTOR_HOSTNAME not set — launching kfutil login" - kfutil login -fi - -kfutil store-types create --name "GcpCertMgr" - -echo "Done. All store types created." diff --git a/scripts/store_types/powershell/kfutil_create_store_types.ps1 b/scripts/store_types/powershell/kfutil_create_store_types.ps1 index 9043b32..67144f8 100644 --- a/scripts/store_types/powershell/kfutil_create_store_types.ps1 +++ b/scripts/store_types/powershell/kfutil_create_store_types.ps1 @@ -1,29 +1,6 @@ -# Creates all 1 store types using kfutil. -# kfutil reads definitions from the Keyfactor integration catalog. -# -# Auth environment variables (first matching method is used): -# OAuth access token: KEYFACTOR_AUTH_ACCESS_TOKEN -# OAuth client creds: KEYFACTOR_AUTH_CLIENT_ID + KEYFACTOR_AUTH_CLIENT_SECRET -# + KEYFACTOR_AUTH_TOKEN_URL -# Basic auth (AD): KEYFACTOR_HOSTNAME + KEYFACTOR_USERNAME + KEYFACTOR_PASSWORD -# + KEYFACTOR_DOMAIN -# -# Auto-generated by doctool generate-store-type-scripts — do not edit by hand. +# Store Type creation script using kfutil +# Generated by Doctool -# Uncomment if kfutil is not in your PATH -# Set-Alias -Name kfutil -Value 'C:\Program Files\Keyfactor\kfutil\kfutil.exe' +Write-Host "Creating store type: GcpCertMgr" +kfutil store-types create GcpCertMgr -if ($null -eq (Get-Command "kfutil" -ErrorAction SilentlyContinue)) { - Write-Host "kfutil could not be found. Please install kfutil" - Write-Host "See https://github.com/Keyfactor/kfutil#quickstart" - exit 1 -} - -if (-not $env:KEYFACTOR_HOSTNAME) { - Write-Host "KEYFACTOR_HOSTNAME not set — launching kfutil login" - & kfutil login -} - -& kfutil store-types create --name "GcpCertMgr" - -Write-Host "Done. All store types created." diff --git a/scripts/store_types/powershell/restmethod_create_store_types.ps1 b/scripts/store_types/powershell/restmethod_create_store_types.ps1 index 8108a7a..c80cbdd 100644 --- a/scripts/store_types/powershell/restmethod_create_store_types.ps1 +++ b/scripts/store_types/powershell/restmethod_create_store_types.ps1 @@ -1,70 +1,19 @@ -# Creates all 1 store types via the Keyfactor Command REST API -# using PowerShell Invoke-RestMethod. -# -# Authentication (first matching method is used): -# OAuth access token: KEYFACTOR_AUTH_ACCESS_TOKEN -# OAuth client creds: KEYFACTOR_AUTH_CLIENT_ID + KEYFACTOR_AUTH_CLIENT_SECRET -# + KEYFACTOR_AUTH_TOKEN_URL -# Basic auth (AD): KEYFACTOR_USERNAME + KEYFACTOR_PASSWORD + KEYFACTOR_DOMAIN -# -# Always required: -# KEYFACTOR_HOSTNAME Command hostname (e.g. my-command.example.com) -# -# Auto-generated by doctool generate-store-type-scripts — do not edit by hand. +# Store Type creation script using Invoke-RestMethod +# Generated by Doctool -if (-not $env:KEYFACTOR_HOSTNAME) { - Write-Error "KEYFACTOR_HOSTNAME is required" - exit 1 -} - -$uri = "https://$($env:KEYFACTOR_HOSTNAME)/keyfactorapi/certificatestoretypes" -$headers = @{ - 'Content-Type' = "application/json" - 'x-keyfactor-requested-with' = "APIClient" -} - -# --------------------------------------------------------------------------- -# Resolve auth -# --------------------------------------------------------------------------- -if ($env:KEYFACTOR_AUTH_ACCESS_TOKEN) { - $headers['Authorization'] = "Bearer $($env:KEYFACTOR_AUTH_ACCESS_TOKEN)" -} elseif ($env:KEYFACTOR_AUTH_CLIENT_ID -and $env:KEYFACTOR_AUTH_CLIENT_SECRET -and $env:KEYFACTOR_AUTH_TOKEN_URL) { - Write-Host "Fetching OAuth token..." - $tokenBody = @{ - grant_type = 'client_credentials' - client_id = $env:KEYFACTOR_AUTH_CLIENT_ID - client_secret = $env:KEYFACTOR_AUTH_CLIENT_SECRET - } - $tokenResp = Invoke-RestMethod -Method Post -Uri $env:KEYFACTOR_AUTH_TOKEN_URL -Body $tokenBody - $headers['Authorization'] = "Bearer $($tokenResp.access_token)" -} elseif ($env:KEYFACTOR_USERNAME -and $env:KEYFACTOR_PASSWORD -and $env:KEYFACTOR_DOMAIN) { - $cred = [System.Convert]::ToBase64String( - [System.Text.Encoding]::ASCII.GetBytes( - "$($env:KEYFACTOR_USERNAME)@$($env:KEYFACTOR_DOMAIN):$($env:KEYFACTOR_PASSWORD)")) - $headers['Authorization'] = "Basic $cred" -} else { - Write-Error ("Authentication required. Set one of:`n" + - " KEYFACTOR_AUTH_ACCESS_TOKEN`n" + - " KEYFACTOR_AUTH_CLIENT_ID + KEYFACTOR_AUTH_CLIENT_SECRET + KEYFACTOR_AUTH_TOKEN_URL`n" + - " KEYFACTOR_USERNAME + KEYFACTOR_PASSWORD + KEYFACTOR_DOMAIN") - exit 1 -} +# Configuration - set these variables before running +$KeyfactorHostname = $env:KEYFACTOR_HOSTNAME +$KeyfactorApiPath = if ($env:KEYFACTOR_API_PATH) { $env:KEYFACTOR_API_PATH } else { "KeyfactorAPI" } +$KeyfactorAuthToken = $env:KEYFACTOR_AUTH_TOKEN -function New-StoreType { - param([string]$Name, [string]$Body) - Write-Host "Creating $Name store type..." - try { - Invoke-RestMethod -Method Post -Uri $uri -Headers $headers -Body $Body -ContentType "application/json" | Out-Null - Write-Host " OK" - } catch { - Write-Warning " FAILED: $($_.Exception.Message)" - } +$Headers = @{ + "Authorization" = "Bearer $KeyfactorAuthToken" + "Content-Type" = "application/json" + "x-keyfactor-requested-with" = "APIClient" } -# --------------------------------------------------------------------------- -# GcpCertMgr — GCP Project ID for your account. -# --------------------------------------------------------------------------- -New-StoreType "GcpCertMgr" @' +Write-Host "Creating store type: GcpCertMgr" +$Body = @' { "Name": "GCP Certificate Manager", "ShortName": "GcpCertMgr", @@ -75,10 +24,10 @@ New-StoreType "GcpCertMgr" @' "PowerShell": false, "PrivateKeyAllowed": "Required", "StorePathType": "", - "StorePathValue": "n/a", + "StorePathValue": "", "SupportedOperations": { "Add": true, - "Create": true, + "Create": false, "Discovery": true, "Enrollment": false, "Remove": true @@ -91,27 +40,44 @@ New-StoreType "GcpCertMgr" @' "Properties": [ { "Name": "Location", - "DisplayName": "Location", + "DisplayName": "Location (deprecated)", "Type": "String", "DependsOn": "", - "DefaultValue": "global", - "Required": true, - "IsPAMEligible": false + "DefaultValue": "", + "Required": false, + "IsPAMEligible": false, + "Description": "**Deprecated in v1.2.** The GCP location is parsed from Store Path. Leave blank for new stores. v1.1-shape stores (where Store Path is blank or `n/a`) still read this value as a fallback; expect a deprecation warning in the orchestrator log when that path is used." }, { "Name": "ServiceAccountKey", - "DisplayName": "Service Account Key File Path", + "DisplayName": "Service Account Key File Path (deprecated)", "Type": "String", "DependsOn": "", "DefaultValue": "", "Required": false, - "IsPAMEligible": false + "IsPAMEligible": false, + "Description": "**Deprecated in v1.2.** Leave blank. Authenticate via Application Default Credentials instead (set `GOOGLE_APPLICATION_CREDENTIALS` as a machine-level environment variable on the orchestrator host pointing at the JSON key, or run on a GCE VM / GKE pod with workload identity). The Discovery job has no way to surface this custom property in Keyfactor Command's discovery-job UI, so ADC is the only mechanism that works uniformly across all four job types. v1.1 stores that have this populated continue to work via a deprecation-logged fallback; the field is scheduled for removal in v2.0." } ], - "StorePathDescription": "This is not used and should be defaulted to n/a per the certificate store type set up.", - "EntryParameters": [] + "EntryParameters": [ + { + "Name": "Scope", + "DisplayName": "Certificate Scope", + "Type": "MultipleChoice", + "DependsOn": "", + "DefaultValue": "DEFAULT", + "Options": "DEFAULT,ALL_REGIONS,EDGE_CACHE,CLIENT_AUTH", + "RequiredWhen": { + "HasPrivateKey": false, + "OnAdd": false, + "OnRemove": false, + "OnReenrollment": false + }, + "Description": "GCP Certificate Manager `scope` for this certificate entry. Allowed: `DEFAULT` (global external Application Load Balancers), `ALL_REGIONS` (cross-region internal Application Load Balancers), `EDGE_CACHE` (Media CDN), `CLIENT_AUTH` (mTLS trust configs / authorized client server certs). **Immutable in GCP** - once a certificate is created with a given scope, GCP refuses to change it. Inventory persists the existing scope back from GCP so renewals carry it forward automatically. A single store can hold certs at different scopes (the field is per-entry, not store-wide)." + } + ] } '@ +Invoke-RestMethod -Uri "https://$KeyfactorHostname/$KeyfactorApiPath/CertificateStoreTypes" -Method POST -Headers $Headers -Body $Body -Write-Host "Completed."