Skip to content

infra: CPU throttling + 1 vCPU API for cost reduction#124

Merged
Pyronewbic merged 1 commit into
mainfrom
dev
May 20, 2026
Merged

infra: CPU throttling + 1 vCPU API for cost reduction#124
Pyronewbic merged 1 commit into
mainfrom
dev

Conversation

@Pyronewbic
Copy link
Copy Markdown
Owner

@Pyronewbic Pyronewbic commented May 20, 2026

Summary

  • Enable cpu_idle (CPU only allocated during requests) on all Cloud Run services
  • Enable startup_cpu_boost to compensate for cold start latency
  • Drop API from 2 vCPU to 1 vCPU (grading calls are I/O-bound)
  • Expected ~60-70% Cloud Run cost reduction

Test plan

  • Terraform plan shows only resource changes (no destroys)
  • Post-deploy health check passes on both regions
  • Search + grading latency unchanged (I/O-bound, not CPU-bound)

@Pyronewbic
infra: enable CPU throttling, drop API to 1 vCPU
046abdf 
Cloud Run was running cpu-throttling: false (CPU always allocated) on
all 4 services across 2 regions — paying for idle vCPUs. Enable
cpu_idle + startup_cpu_boost on both API and site services. Drop API
from 2 vCPU to 1 vCPU (LLM grading is I/O-bound, not CPU-bound).
Expected ~60-70% Cloud Run cost reduction.
@github-actions
Copy link
Copy Markdown

github-actions Bot commented May 20, 2026

Terraform Plan

Acquiring state lock. This may take a few moments...
data.google_secret_manager_secret_version.api_key: Reading...
google_project_service.binaryauthorization: Refreshing state... [id=casecomp-495718/binaryauthorization.googleapis.com]
google_project_service.secretmanager: Refreshing state... [id=casecomp-495718/secretmanager.googleapis.com]
google_project_service.containeranalysis: Refreshing state... [id=casecomp-495718/containeranalysis.googleapis.com]
google_project_service.compute: Refreshing state... [id=casecomp-495718/compute.googleapis.com]
google_project_service.scheduler: Refreshing state... [id=casecomp-495718/cloudscheduler.googleapis.com]
google_logging_metric.api_errors: Refreshing state... [id=cardscrapebot-errors]
google_project_service.run: Refreshing state... [id=casecomp-495718/run.googleapis.com]
google_project_service.cloudbuild: Refreshing state... [id=casecomp-495718/cloudbuild.googleapis.com]
google_storage_bucket.site: Refreshing state... [id=casecomp-site]
data.google_project.current: Reading...
google_project_service.monitoring: Refreshing state... [id=casecomp-495718/monitoring.googleapis.com]
data.google_secret_manager_secret_version.api_key: Read complete after 1s [id=projects/129850122606/secrets/CASECOMP_API_KEY/versions/1]
google_compute_managed_ssl_certificate.site_cert: Refreshing state... [id=projects/casecomp-495718/global/sslCertificates/casecomp-site-cert]
google_compute_managed_ssl_certificate.api_cert: Refreshing state... [id=projects/casecomp-495718/global/sslCertificates/cardscrapebot-cert-v2]
data.google_project.current: Read complete after 0s [id=projects/casecomp-495718]
google_project_service.firestore: Refreshing state... [id=casecomp-495718/firestore.googleapis.com]
google_storage_bucket_iam_member.site_public: Refreshing state... [id=b/casecomp-site/roles/storage.objectViewer/allUsers]
google_compute_global_address.api_ip: Refreshing state... [id=projects/casecomp-495718/global/addresses/cardscrapebot-ip]
google_binary_authorization_policy.default: Refreshing state... [id=projects/casecomp-495718]
google_cloud_scheduler_job.track_prices: Refreshing state... [id=projects/casecomp-495718/locations/asia-south1/jobs/casecomp-track-prices]
google_secret_manager_secret.api_secrets["RESEND_API_KEY"]: Refreshing state... [id=projects/casecomp-495718/secrets/RESEND_API_KEY]
google_secret_manager_secret.api_secrets["CASECOMP_ADMIN_SUB"]: Refreshing state... [id=projects/casecomp-495718/secrets/CASECOMP_ADMIN_SUB]
google_secret_manager_secret.api_secrets["CASECOMP_API_KEY"]: Refreshing state... [id=projects/casecomp-495718/secrets/CASECOMP_API_KEY]
google_secret_manager_secret.api_secrets["EBAY_CLIENT_SECRET"]: Refreshing state... [id=projects/casecomp-495718/secrets/EBAY_CLIENT_SECRET]
google_secret_manager_secret.api_secrets["PSA_AUTH_TOKEN"]: Refreshing state... [id=projects/casecomp-495718/secrets/PSA_AUTH_TOKEN]
google_secret_manager_secret.api_secrets["ANTHROPIC_API_KEY"]: Refreshing state... [id=projects/casecomp-495718/secrets/ANTHROPIC_API_KEY]
google_secret_manager_secret.api_secrets["CASECOMP_SANDBOX_KEY"]: Refreshing state... [id=projects/casecomp-495718/secrets/CASECOMP_SANDBOX_KEY]
google_secret_manager_secret.api_secrets["GOOGLE_OAUTH_CLIENT_ID"]: Refreshing state... [id=projects/casecomp-495718/secrets/GOOGLE_OAUTH_CLIENT_ID]
google_secret_manager_secret.api_secrets["EBAY_CLIENT_ID"]: Refreshing state... [id=projects/casecomp-495718/secrets/EBAY_CLIENT_ID]
google_secret_manager_secret.api_secrets["TOGETHER_API_KEY"]: Refreshing state... [id=projects/casecomp-495718/secrets/TOGETHER_API_KEY]
google_secret_manager_secret.api_secrets["CASECOMP_JWT_SECRET"]: Refreshing state... [id=projects/casecomp-495718/secrets/CASECOMP_JWT_SECRET]
google_cloud_scheduler_job.check_alerts: Refreshing state... [id=projects/casecomp-495718/locations/asia-south1/jobs/casecomp-check-alerts]
google_monitoring_uptime_check_config.api_uptime: Refreshing state... [id=projects/casecomp-495718/uptimeCheckConfigs/casecomp-api-health-lQkUaC0Vzb8]
google_firestore_database.default: Refreshing state... [id=projects/casecomp-495718/databases/(default)]
google_monitoring_notification_channel.email: Refreshing state... [id=projects/casecomp-495718/notificationChannels/3431772178774051140]
google_cloud_run_v2_service.site["us-central1"]: Refreshing state... [id=projects/casecomp-495718/locations/us-central1/services/casecomp-site]
google_cloud_run_v2_service.site["asia-south1"]: Refreshing state... [id=projects/casecomp-495718/locations/asia-south1/services/casecomp-site]
google_secret_manager_secret_iam_member.cloud_run_access["EBAY_CLIENT_ID"]: Refreshing state... [id=projects/casecomp-495718/secrets/EBAY_CLIENT_ID/roles/secretmanager.secretAccessor/serviceAccount:129850122606-compute@developer.gserviceaccount.com]
google_secret_manager_secret_iam_member.cloud_run_access["CASECOMP_API_KEY"]: Refreshing state... [id=projects/casecomp-495718/secrets/CASECOMP_API_KEY/roles/secretmanager.secretAccessor/serviceAccount:129850122606-compute@developer.gserviceaccount.com]
google_secret_manager_secret_iam_member.cloud_run_access["ANTHROPIC_API_KEY"]: Refreshing state... [id=projects/casecomp-495718/secrets/ANTHROPIC_API_KEY/roles/secretmanager.secretAccessor/serviceAccount:129850122606-compute@developer.gserviceaccount.com]
google_secret_manager_secret_iam_member.cloud_run_access["CASECOMP_JWT_SECRET"]: Refreshing state... [id=projects/casecomp-495718/secrets/CASECOMP_JWT_SECRET/roles/secretmanager.secretAccessor/serviceAccount:129850122606-compute@developer.gserviceaccount.com]
google_secret_manager_secret_iam_member.cloud_run_access["RESEND_API_KEY"]: Refreshing state... [id=projects/casecomp-495718/secrets/RESEND_API_KEY/roles/secretmanager.secretAccessor/serviceAccount:129850122606-compute@developer.gserviceaccount.com]
google_secret_manager_secret_iam_member.cloud_run_access["PSA_AUTH_TOKEN"]: Refreshing state... [id=projects/casecomp-495718/secrets/PSA_AUTH_TOKEN/roles/secretmanager.secretAccessor/serviceAccount:129850122606-compute@developer.gserviceaccount.com]
google_secret_manager_secret_iam_member.cloud_run_access["EBAY_CLIENT_SECRET"]: Refreshing state... [id=projects/casecomp-495718/secrets/EBAY_CLIENT_SECRET/roles/secretmanager.secretAccessor/serviceAccount:129850122606-compute@developer.gserviceaccount.com]
google_secret_manager_secret_iam_member.cloud_run_access["GOOGLE_OAUTH_CLIENT_ID"]: Refreshing state... [id=projects/casecomp-495718/secrets/GOOGLE_OAUTH_CLIENT_ID/roles/secretmanager.secretAccessor/serviceAccount:129850122606-compute@developer.gserviceaccount.com]
google_secret_manager_secret_iam_member.cloud_run_access["CASECOMP_ADMIN_SUB"]: Refreshing state... [id=projects/casecomp-495718/secrets/CASECOMP_ADMIN_SUB/roles/secretmanager.secretAccessor/serviceAccount:129850122606-compute@developer.gserviceaccount.com]
google_secret_manager_secret_iam_member.cloud_run_access["TOGETHER_API_KEY"]: Refreshing state... [id=projects/casecomp-495718/secrets/TOGETHER_API_KEY/roles/secretmanager.secretAccessor/serviceAccount:129850122606-compute@developer.gserviceaccount.com]
google_secret_manager_secret_iam_member.cloud_run_access["CASECOMP_SANDBOX_KEY"]: Refreshing state... [id=projects/casecomp-495718/secrets/CASECOMP_SANDBOX_KEY/roles/secretmanager.secretAccessor/serviceAccount:129850122606-compute@developer.gserviceaccount.com]
google_monitoring_alert_policy.api_error_alert: Refreshing state... [id=projects/casecomp-495718/alertPolicies/16365448047387079183]
google_monitoring_alert_policy.api_uptime_alert: Refreshing state... [id=projects/casecomp-495718/alertPolicies/14098674883088940398]
google_cloud_run_v2_service.api["us-central1"]: Refreshing state... [id=projects/casecomp-495718/locations/us-central1/services/casecomp-api]
google_cloud_run_v2_service.api["asia-south1"]: Refreshing state... [id=projects/casecomp-495718/locations/asia-south1/services/casecomp-api]
google_firestore_index.composite["grade-logs_source_createdAt"]: Refreshing state... [id=projects/casecomp-495718/databases/(default)/collectionGroups/grade-logs/indexes/CICAgJj7z4EJ]
google_firestore_index.composite["api-analytics_userId_ts"]: Refreshing state... [id=projects/casecomp-495718/databases/(default)/collectionGroups/api-analytics/indexes/CICAgJjF9oIK]
google_firestore_index.composite["grade-logs_userId_createdAt"]: Refreshing state... [id=projects/casecomp-495718/databases/(default)/collectionGroups/grade-logs/indexes/CICAgJim14AK]
google_firestore_index.composite["price-history_cardKey_recordedAt"]: Refreshing state... [id=projects/casecomp-495718/databases/(default)/collectionGroups/price-history/indexes/CICAgOjXh4EK]
google_firestore_index.composite["api-keys_ownerId_createdAt"]: Refreshing state... [id=projects/casecomp-495718/databases/(default)/collectionGroups/api-keys/indexes/CICAgJiUpoMK]
google_compute_region_network_endpoint_group.site_neg["asia-south1"]: Refreshing state... [id=projects/casecomp-495718/regions/asia-south1/networkEndpointGroups/casecomp-site-neg]
google_compute_region_network_endpoint_group.site_neg["us-central1"]: Refreshing state... [id=projects/casecomp-495718/regions/us-central1/networkEndpointGroups/casecomp-site-neg-us-central1]
google_cloud_run_v2_service_iam_member.site_public["us-central1"]: Refreshing state... [id=projects/casecomp-495718/locations/us-central1/services/casecomp-site/roles/run.invoker/allUsers]
google_cloud_run_v2_service_iam_member.site_public["asia-south1"]: Refreshing state... [id=projects/casecomp-495718/locations/asia-south1/services/casecomp-site/roles/run.invoker/allUsers]
google_compute_region_network_endpoint_group.api_neg["us-central1"]: Refreshing state... [id=projects/casecomp-495718/regions/us-central1/networkEndpointGroups/casecomp-api-neg-us-central1]
google_compute_region_network_endpoint_group.api_neg["asia-south1"]: Refreshing state... [id=projects/casecomp-495718/regions/asia-south1/networkEndpointGroups/casecomp-api-neg]
google_cloud_run_v2_service_iam_member.public["asia-south1"]: Refreshing state... [id=projects/casecomp-495718/locations/asia-south1/services/casecomp-api/roles/run.invoker/allUsers]
google_cloud_run_v2_service_iam_member.public["us-central1"]: Refreshing state... [id=projects/casecomp-495718/locations/us-central1/services/casecomp-api/roles/run.invoker/allUsers]
google_compute_backend_service.site_backend: Refreshing state... [id=projects/casecomp-495718/global/backendServices/casecomp-site-backend]
google_compute_backend_service.api_backend: Refreshing state... [id=projects/casecomp-495718/global/backendServices/cardscrapebot-backend]
google_compute_url_map.api_urlmap: Refreshing state... [id=projects/casecomp-495718/global/urlMaps/cardscrapebot-urlmap]
google_compute_target_https_proxy.api_proxy: Refreshing state... [id=projects/casecomp-495718/global/targetHttpsProxies/cardscrapebot-https-proxy]
google_compute_global_forwarding_rule.api_https: Refreshing state... [id=projects/casecomp-495718/global/forwardingRules/cardscrapebot-https-rule]

Terraform used the selected providers to generate the following execution
plan. Resource actions are indicated with the following symbols:
  ~ update in-place

Terraform will perform the following actions:

  # google_cloud_run_v2_service.api["asia-south1"] will be updated in-place
  ~ resource "google_cloud_run_v2_service" "api" {
        id                      = "projects/casecomp-495718/locations/asia-south1/services/casecomp-api"
        name                    = "casecomp-api"
        # (30 unchanged attributes hidden)

      ~ template {
            # (9 unchanged attributes hidden)

          ~ containers {
                name        = null
                # (5 unchanged attributes hidden)

              ~ resources {
                  ~ cpu_idle          = false -> true
                  ~ limits            = {
                      ~ "cpu"    = "2000m" -> "1000m"
                        # (1 unchanged element hidden)
                    }
                  ~ startup_cpu_boost = false -> true
                }

                # (14 unchanged blocks hidden)
            }

            # (1 unchanged block hidden)
        }

        # (2 unchanged blocks hidden)
    }

  # google_cloud_run_v2_service.api["us-central1"] will be updated in-place
  ~ resource "google_cloud_run_v2_service" "api" {
        id                      = "projects/casecomp-495718/locations/us-central1/services/casecomp-api"
        name                    = "casecomp-api"
        # (30 unchanged attributes hidden)

      ~ template {
            # (9 unchanged attributes hidden)

          ~ containers {
                name        = null
                # (5 unchanged attributes hidden)

              ~ resources {
                  ~ cpu_idle          = false -> true
                  ~ limits            = {
                      ~ "cpu"    = "2000m" -> "1000m"
                        # (1 unchanged element hidden)
                    }
                  ~ startup_cpu_boost = false -> true
                }

                # (14 unchanged blocks hidden)
            }

            # (1 unchanged block hidden)
        }

        # (2 unchanged blocks hidden)
    }

  # google_cloud_run_v2_service.site["asia-south1"] will be updated in-place
  ~ resource "google_cloud_run_v2_service" "site" {
        id                      = "projects/casecomp-495718/locations/asia-south1/services/casecomp-site"
        name                    = "casecomp-site"
        # (30 unchanged attributes hidden)

      ~ template {
            # (9 unchanged attributes hidden)

          ~ containers {
                name        = null
                # (5 unchanged attributes hidden)

              ~ resources {
                  ~ cpu_idle          = false -> true
                  ~ startup_cpu_boost = false -> true
                    # (1 unchanged attribute hidden)
                }

                # (2 unchanged blocks hidden)
            }

            # (1 unchanged block hidden)
        }

        # (2 unchanged blocks hidden)
    }

  # google_cloud_run_v2_service.site["us-central1"] will be updated in-place
  ~ resource "google_cloud_run_v2_service" "site" {
        id                      = "projects/casecomp-495718/locations/us-central1/services/casecomp-site"
        name                    = "casecomp-site"
        # (30 unchanged attributes hidden)

      ~ template {
            # (9 unchanged attributes hidden)

          ~ containers {
                name        = null
                # (5 unchanged attributes hidden)

              ~ resources {
                  ~ cpu_idle          = false -> true
                  ~ startup_cpu_boost = false -> true
                    # (1 unchanged attribute hidden)
                }

                # (2 unchanged blocks hidden)
            }

            # (1 unchanged block hidden)
        }

        # (2 unchanged blocks hidden)
    }

Plan: 0 to add, 4 to change, 0 to destroy.

─────────────────────────────────────────────────────────────────────────────

Saved the plan to: tfplan

To perform exactly these actions, run the following command to apply:
    terraform apply "tfplan"

Merge to main to apply.

@Pyronewbic Pyronewbic merged commit e2bc06f into main May 20, 2026
18 of 19 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@Pyronewbic