Skip to content

OCPBUGS-99755: OTE: fix AWS endpoint resolution for non-standard partitions#494

Open
tthvo wants to merge 1 commit into
openshift:mainfrom
tthvo:OCPBUGS-99755
Open

OCPBUGS-99755: OTE: fix AWS endpoint resolution for non-standard partitions#494
tthvo wants to merge 1 commit into
openshift:mainfrom
tthvo:OCPBUGS-99755

Conversation

@tthvo

@tthvo tthvo commented Jul 24, 2026

Copy link
Copy Markdown
Member

Description

The BaseEndpoint for ELBv2 and EC2 clients was hardcoded to .amazonaws.com, which is incorrect for non-standard AWS partitions like EUSC (.amazonaws.eu). This caused API calls to fail on EUSC clusters because the requests were sent to the wrong domain.

This PR introduces the use of partition-aware DNS suffix resolution instead of hardcoding .amazonaws.com, so that all AWS partitions (EUSC, China, GovCloud, ISO, etc.) resolve to the correct (public) regional endpoint.

See #464 for why we need to set the endpoint for AWS clients.

Originally, I thought it was due to an outdated AWS SDK deps, but the endpoint resolution is fine (without base endpoint override) thanks to the compatible core config module. Thus, there is no longer any need to upgrade go deps.

Summary by CodeRabbit

  • Bug Fixes
    • Improved AWS endpoint handling across regional partitions.
    • Ensured connections use the correct DNS suffix for standard, China, European Sovereign, and isolated AWS regions.

@openshift-ci-robot openshift-ci-robot added jira/valid-reference Indicates that this PR references a valid Jira ticket of any type. jira/invalid-bug Indicates that a referenced Jira bug is invalid for the branch this PR is targeting. labels Jul 24, 2026
@openshift-ci-robot

Copy link
Copy Markdown

@tthvo: This pull request references Jira Issue OCPBUGS-99755, which is invalid:

  • expected the bug to target the "5.0.0" version, but no target version was set

Comment /jira refresh to re-evaluate validity if changes to the Jira bug are made, or edit the title of this pull request to link to a different bug.

The bug has been updated to refer to the pull request using the external bug tracker.

Details

In response to this:

Description

NLB Security Group OTEs are failing on the installer's e2e-aws-eusc-techpreview presubmit (see example) due to invalid API endpoint resolution. For EUSC, the endpoints have .eu top-level domain instead of .com.

I0724 01:09:00.576463  158635 helper.go:78] transient error describing load balancers (will retry): operation error Elastic Load Balancing v2: DescribeLoadBalancers, https response error StatusCode: 0, RequestID: , request send failed, Post "https://elasticloadbalancing.eusc-de-east-1.amazonaws.com/": dial tcp: lookup elasticloadbalancing.eusc-de-east-1.amazonaws.com on 172.30.0.10:53: no such host

This PR bumps github.com/aws/aws-sdk-go-v2/service/elasticloadbalancingv2 to get EUSC endpoint support.

Notes

Claude analysis shows that EUSC support is available since version:

┌──────────────────────────────────────┬─────────────────────────┬─────────────────────────────────────┐
│               Package                │ First version with EUSC │                Date                 │
├──────────────────────────────────────┼─────────────────────────┼─────────────────────────────────────┤
│ aws-sdk-go-v2 (core partitions.json) │ v1.41.2                 │ 2026-02-23 (commit from 2026-01-15) │
├──────────────────────────────────────┼─────────────────────────┼─────────────────────────────────────┤
│ service/ec2                          │ v1.211.1                │ 2025-04-03                          │
├──────────────────────────────────────┼─────────────────────────┼─────────────────────────────────────┤
│ service/elasticloadbalancingv2       │ v1.54.5                 │ 2025-12-09                          │
└──────────────────────────────────────┴─────────────────────────┴─────────────────────────────────────┘

I took this chance to upgrade service/elasticloadbalancingv2 to latest v1.58.1 while keeping service/ec2 "as-is" to minimize the vendor change.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

@openshift-ci
openshift-ci Bot requested a review from damdo July 24, 2026 20:01
@openshift-ci

openshift-ci Bot commented Jul 24, 2026

Copy link
Copy Markdown
Contributor

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by:
Once this PR has been reviewed and has the lgtm label, please assign mdbooth for approval. For more information see the Code Review Process.

The full list of commands accepted by this bot can be found here.

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@openshift-ci
openshift-ci Bot requested a review from theobarberbany July 24, 2026 20:01
@coderabbitai

coderabbitai Bot commented Jul 24, 2026

Copy link
Copy Markdown

Walkthrough

AWS ELBv2 and EC2 client endpoints now derive region-specific DNS suffixes through a shared helper, supporting standard and non-standard AWS partitions while retaining amazonaws.com as the default.

Changes

AWS endpoint partition handling

Layer / File(s) Summary
Partition-aware regional endpoints
openshift-tests/ccm-aws-tests/e2e/aws/helper.go
dnsSuffixForRegion maps region prefixes to partition DNS suffixes, and ELBv2 and EC2 BaseEndpoint values use the selected suffix.

Estimated code review effort: 2 (Simple) | ~10 minutes

Suggested reviewers: mtulio, damdo, mfbonfigli, theobarberbany

🚥 Pre-merge checks | ✅ 15
✅ Passed checks (15 passed)
Check name Status Explanation
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Stable And Deterministic Test Names ✅ Passed The PR only changes helper.go endpoint construction; no Ginkgo test titles were added or modified, so the check is not applicable.
Test Structure And Quality ✅ Passed PR only changes AWS client endpoint helpers; no Ginkgo test blocks or wait/cleanup structure were modified, and existing specs already use timeouts and cleanup.
Microshift Test Compatibility ✅ Passed Only helper.go changed; no new Ginkgo tests or OpenShift API/resource usage were added, so MicroShift compatibility is unaffected.
Single Node Openshift (Sno) Test Compatibility ✅ Passed PASS — The PR only changes AWS endpoint helper code in helper.go; no new Ginkgo tests or SNO-relevant assumptions were added.
Topology-Aware Scheduling Compatibility ✅ Passed Only AWS client endpoint helpers changed; no manifests, controllers, replicas, affinity, or node selection logic were modified.
Ote Binary Stdout Contract ✅ Passed Diff only changes AWS endpoint construction and adds dnsSuffixForRegion; no main/init/suite-level stdout writes were added.
Ipv6 And Disconnected Network Test Compatibility ✅ Passed No new Ginkgo tests were added; the diff only updates AWS endpoint suffix mapping and adds no IPv4-specific parsing or localhost assumptions.
No-Weak-Crypto ✅ Passed Patch only changes AWS endpoint suffix construction; no weak crypto, custom crypto, or secret comparisons were added.
Container-Privileges ✅ Passed Only helper.go changed; it adds AWS endpoint suffix logic and contains no privileged/container securityContext settings or K8s manifests.
No-Sensitive-Data-In-Logs ✅ Passed PASS: The patch only changes AWS endpoint construction; it adds no new log statements or sensitive-data emissions.
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly matches the main change: fixing AWS endpoint resolution for non-standard partitions.
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Comment @coderabbitai help to get the list of available commands.

@tthvo

tthvo commented Jul 24, 2026

Copy link
Copy Markdown
Member Author

/jira refresh

@openshift-ci-robot openshift-ci-robot added jira/valid-bug Indicates that a referenced Jira bug is valid for the branch this PR is targeting. and removed jira/invalid-bug Indicates that a referenced Jira bug is invalid for the branch this PR is targeting. labels Jul 24, 2026
@openshift-ci-robot

Copy link
Copy Markdown

@tthvo: This pull request references Jira Issue OCPBUGS-99755, which is valid. The bug has been moved to the POST state.

3 validation(s) were run on this bug
  • bug is open, matching expected state (open)
  • bug target version (5.0.0) matches configured target version for branch (5.0.0)
  • bug is in the state New, which is one of the valid states (NEW, ASSIGNED, POST)
Details

In response to this:

/jira refresh

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

@tthvo

tthvo commented Jul 24, 2026

Copy link
Copy Markdown
Member Author

/cc @mtulio @mfbonfigli

@openshift-ci
openshift-ci Bot requested review from mfbonfigli and mtulio July 24, 2026 20:02
@openshift-ci-robot

Copy link
Copy Markdown

@tthvo: This pull request references Jira Issue OCPBUGS-99755, which is valid.

3 validation(s) were run on this bug
  • bug is open, matching expected state (open)
  • bug target version (5.0.0) matches configured target version for branch (5.0.0)
  • bug is in the state POST, which is one of the valid states (NEW, ASSIGNED, POST)
Details

In response to this:

Description

NLB Security Group OTEs are failing on the installer's e2e-aws-eusc-techpreview presubmit (see example) due to invalid API endpoint resolution. For EUSC, the endpoints have .eu top-level domain instead of .com.

I0724 01:09:00.576463  158635 helper.go:78] transient error describing load balancers (will retry): operation error Elastic Load Balancing v2: DescribeLoadBalancers, https response error StatusCode: 0, RequestID: , request send failed, Post "https://elasticloadbalancing.eusc-de-east-1.amazonaws.com/": dial tcp: lookup elasticloadbalancing.eusc-de-east-1.amazonaws.com on 172.30.0.10:53: no such host

This PR bumps github.com/aws/aws-sdk-go-v2/service/elasticloadbalancingv2 to get EUSC endpoint support.

Notes

Claude analysis shows that EUSC support is available since version:

┌──────────────────────────────────────┬─────────────────────────┬─────────────────────────────────────┐
│               Package                │ First version with EUSC │                Date                 │
├──────────────────────────────────────┼─────────────────────────┼─────────────────────────────────────┤
│ aws-sdk-go-v2 (core partitions.json) │ v1.41.2                 │ 2026-02-23 (commit from 2026-01-15) │
├──────────────────────────────────────┼─────────────────────────┼─────────────────────────────────────┤
│ service/ec2                          │ v1.211.1                │ 2025-04-03                          │
├──────────────────────────────────────┼─────────────────────────┼─────────────────────────────────────┤
│ service/elasticloadbalancingv2       │ v1.54.5                 │ 2025-12-09                          │
└──────────────────────────────────────┴─────────────────────────┴─────────────────────────────────────┘

I took this chance to upgrade service/elasticloadbalancingv2 to latest v1.58.1 while keeping service/ec2 "as-is" to minimize the vendor change.

Summary by CodeRabbit

  • Chores
  • Updated AWS SDK components used by the AWS integration tests.
  • Upgraded supporting AWS configuration, endpoint, and request-handling libraries for improved compatibility.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

@tthvo

tthvo commented Jul 24, 2026

Copy link
Copy Markdown
Member Author

/testwith openshift/installer/main/e2e-aws-eusc-techpreview openshift/installer#10672

@tthvo

tthvo commented Jul 24, 2026

Copy link
Copy Markdown
Member Author

/payload-job periodic-ci-openshift-hypershift-release-5.0-periodics-e2e-aws-ovn-conformance

@openshift-ci

openshift-ci Bot commented Jul 24, 2026

Copy link
Copy Markdown
Contributor

@tthvo: trigger 1 job(s) for the /payload-(with-prs|job|aggregate|job-with-prs|aggregate-with-prs) command

  • periodic-ci-openshift-hypershift-release-5.0-periodics-e2e-aws-ovn-conformance

See details on https://pr-payload-tests.ci.openshift.org/runs/ci/840867a0-879b-11f1-8fa2-ca425d91c641-0

The BaseEndpoint for ELBv2 and EC2 clients was hardcoded to
.amazonaws.com, which is incorrect for non-standard AWS partitions
like EUSC (.amazonaws.eu). This caused API calls to fail on EUSC
clusters because the requests were sent to the wrong domain.

Use partition-aware DNS suffix resolution instead of hardcoding
.amazonaws.com, so that all AWS partitions (EUSC, China, GovCloud,
ISO, etc.) resolve to the correct public regional endpoint.
@tthvo tthvo changed the title OCPBUGS-99755: OTE: bump elasticloadbalancingv2 to support AWS EUSC OCPBUGS-99755: OTE: fix AWS endpoint resolution for non-standard partitions Jul 24, 2026
@openshift-ci-robot

Copy link
Copy Markdown

@tthvo: This pull request references Jira Issue OCPBUGS-99755, which is valid.

3 validation(s) were run on this bug
  • bug is open, matching expected state (open)
  • bug target version (5.0.0) matches configured target version for branch (5.0.0)
  • bug is in the state POST, which is one of the valid states (NEW, ASSIGNED, POST)

The bug has been updated to refer to the pull request using the external bug tracker.

Details

In response to this:

Description

NLB Security Group OTEs are failing on the installer's e2e-aws-eusc-techpreview presubmit (see example) due to invalid API endpoint resolution. For EUSC, the endpoints have .eu top-level domain instead of .com.

I0724 01:09:00.576463  158635 helper.go:78] transient error describing load balancers (will retry): operation error Elastic Load Balancing v2: DescribeLoadBalancers, https response error StatusCode: 0, RequestID: , request send failed, Post "https://elasticloadbalancing.eusc-de-east-1.amazonaws.com/": dial tcp: lookup elasticloadbalancing.eusc-de-east-1.amazonaws.com on 172.30.0.10:53: no such host

This PR bumps github.com/aws/aws-sdk-go-v2/service/elasticloadbalancingv2 to get EUSC endpoint support.

Notes

Claude analysis shows that EUSC support is available since version:

┌──────────────────────────────────────┬─────────────────────────┬─────────────────────────────────────┐
│               Package                │ First version with EUSC │                Date                 │
├──────────────────────────────────────┼─────────────────────────┼─────────────────────────────────────┤
│ aws-sdk-go-v2 (core partitions.json) │ v1.41.2                 │ 2026-02-23 (commit from 2026-01-15) │
├──────────────────────────────────────┼─────────────────────────┼─────────────────────────────────────┤
│ service/ec2                          │ v1.211.1                │ 2025-04-03                          │
├──────────────────────────────────────┼─────────────────────────┼─────────────────────────────────────┤
│ service/elasticloadbalancingv2       │ v1.54.5                 │ 2025-12-09                          │
└──────────────────────────────────────┴─────────────────────────┴─────────────────────────────────────┘

I took this chance to upgrade service/elasticloadbalancingv2 to latest v1.58.1 while keeping service/ec2 "as-is" to minimize the vendor change.

Summary by CodeRabbit

  • Bug Fixes
  • Improved AWS endpoint handling across regional partitions.
  • Ensured connections use the correct DNS suffix for standard, China, European Sovereign, and isolated AWS regions.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🧹 Nitpick comments (1)
openshift-tests/ccm-aws-tests/e2e/aws/helper.go (1)

200-220: 🎯 Functional Correctness | 🔵 Trivial | ⚡ Quick win

Add table-driven tests for every partition suffix.

This helper is shared by both ELBv2 and EC2 clients; test cn-, eusc-, each ISO prefix, us-gov-, and the default case to prevent silent endpoint regressions.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@openshift-tests/ccm-aws-tests/e2e/aws/helper.go` around lines 200 - 220, Add
table-driven unit tests for dnsSuffixForRegion covering cn-, eusc-, us-iso-,
us-isob-, eu-isoe-, us-isof-, us-gov-, and an unmatched region. Assert each
input returns its expected partition suffix, including the default amazonaws.com
case.

Source: MCP tools

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Nitpick comments:
In `@openshift-tests/ccm-aws-tests/e2e/aws/helper.go`:
- Around line 200-220: Add table-driven unit tests for dnsSuffixForRegion
covering cn-, eusc-, us-iso-, us-isob-, eu-isoe-, us-isof-, us-gov-, and an
unmatched region. Assert each input returns its expected partition suffix,
including the default amazonaws.com case.

ℹ️ Review info
⚙️ Run configuration

Configuration used: Repository: openshift/coderabbit/.coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 002f9648-ade4-4030-9da3-1446579091d8

📥 Commits

Reviewing files that changed from the base of the PR and between f3d4f2b and 12feedf.

📒 Files selected for processing (1)
  • openshift-tests/ccm-aws-tests/e2e/aws/helper.go

@tthvo

tthvo commented Jul 24, 2026

Copy link
Copy Markdown
Member Author

/payload-job periodic-ci-openshift-hypershift-release-5.0-periodics-e2e-aws-ovn-conformance

@openshift-ci

openshift-ci Bot commented Jul 24, 2026

Copy link
Copy Markdown
Contributor

@tthvo: trigger 1 job(s) for the /payload-(with-prs|job|aggregate|job-with-prs|aggregate-with-prs) command

  • periodic-ci-openshift-hypershift-release-5.0-periodics-e2e-aws-ovn-conformance

See details on https://pr-payload-tests.ci.openshift.org/runs/ci/6e13f750-87a2-11f1-8221-50acf096a3f1-0

@tthvo

tthvo commented Jul 24, 2026

Copy link
Copy Markdown
Member Author

/testwith openshift/installer/main/e2e-aws-eusc-techpreview openshift/installer#10672

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

jira/valid-bug Indicates that a referenced Jira bug is valid for the branch this PR is targeting. jira/valid-reference Indicates that this PR references a valid Jira ticket of any type.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants