Skip to content

feat(usagesync): measure Lambda and NAT from CloudWatch (#97) - #8

Open
asakaxgit wants to merge 1 commit into
usage-sync/s3-requestsfrom
usage-sync/requests
Open

asakaxgit wants to merge 1 commit into
usage-sync/s3-requestsfrom
usage-sync/requests

Conversation

@asakaxgit

@asakaxgit asakaxgit commented Oct 28, 2026 •

Copy link
Copy Markdown
Owner

PR 3 of the staged plan in docs/design/usage-sync.md (c3xdev#97): Lambda and NAT gateway quantities from CloudWatch. Stacked on #7 (fork copy of c3xdev#119) so this diff is only this stage.

What it measures

aws_lambda_function, the keys resources/aws/aws_lambda_function.toml reads for invocations and duration:

  • monthly_requests from Invocations (Sum), the count over the window expressed per 30 days. The default --days 30 is the measured total. series is the same rate per calendar month.
  • request_duration_ms from Duration (Sum, milliseconds) divided by Invocations (Sum). That is the invocation-weighted average, and it is not scaled to 30 days. series is the same average for each month. A month with invocations but no Duration points is omitted from the duration series.
  • Dimension FunctionName only. Resource=$LATEST would drop aliases and published versions. InitDuration is cold start and the catalog has no key for it, so it is not added.
  • The two keys are written together or not at all. No Invocations datapoints, or Invocations with no Duration, is an error and neither key is written. The catalog defaults a missing request_duration_ms to 100 ms, which would become a real cost once monthly_requests is set.

aws_nat_gateway, the key resources/aws/aws_nat_gateway.toml reads for data processed:

  • monthly_data_processed_gb from BytesInFromSource + BytesInFromDestination (Sum), converted with 1 GB = 2^30 bytes, then expressed per 30 days. Bytes out the other side are the same flows and are not added. A 1 GB upload is billed as 1 GB.
  • A zonal gateway publishes those metrics with only NatGatewayId. That series is used, and availability zones are not also summed.
  • A regional gateway publishes per availability zone, so the gateway-level query is empty. ListMetrics (no metric name) finds zones whose dimensions are exactly NatGatewayId and AvailabilityZone. Those zones are summed. Any other dimension shape is ignored.
  • No byte datapoints is an error. monthly_inter_az_gb is not emitted: the catalog uses it for a hypothetical consolidation and defaults it to 0.

Names come from state, else a literal: function_name for Lambda, id for a NAT gateway. A NAT id is computed, so without --state the gateway is listed and not guessed.

Checked against the design

  • Only usage sync loads credentials, through the SDK default chain. Calls stay read-only GetMetricData and ListMetrics. No new SDK module: still config and cloudwatch.
  • Estimates do not call a cloud API. Untrusted mode still refuses to run.
  • Nothing is guessed: no 100 ms duration, no zero for a function with no datapoints, no monthly_inter_az_gb, no sum of a zonal total plus its zones, no dimension shape other than the ones above.
  • The hand-written file is still untouched and still wins per key. Writes stay atomic.
  • depguard is unchanged and still holds. Emits includes the new keys, and the catalog-key test checks every one.
  • Tests use a fake CloudWatch client. No cloud calls.

Local checks

go build ./..., go vet ./..., go test ./..., and go test -race ./internal/usagesync/... ./cmd/c3x/ pass. golangci-lint v2.15.0 reports 0 issues.

Lambda writes monthly_requests and request_duration_ms together from
Invocations and Duration, so a missing duration is not filled with the
catalog default. NAT writes monthly_data_processed_gb from bytes in,
summing availability zones only when the gateway-level metric is empty.

Signed-off-by: Atsushi Sakashita <ats@saks.industries>
Copilot AI balanced review requested due to automatic review settings October 28, 2026 05:13
@asakaxgit
asakaxgit added this pull request to stack #9 October 28, 2026 05:13

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has insufficient remaining quota.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants