Skip to content

load-balancer-source-ranges auto-detects NAT Gateway EIP, blocking browser access to ui-nlb #1857

@w7089

Description

@w7089

What happened?

In the fundamentals/exposing/loadbalancer lab, the ui-nlb Service uses:

service.beta.kubernetes.io/load-balancer-source-ranges: $INBOUND_CIDRS

When the workshop environment resolves SOURCE_IP_ADDRESS=auto / INBOUND_CIDRS from inside the workshop/EKS environment, https://checkip.amazonaws.com can return the VPC NAT Gateway Elastic IP rather than the participant browser/client public IP.

In my run, this produced a /32 CIDR for the workshop VPC NAT Gateway EIP rather than my local browser IP. The AWS Load Balancer Controller then created the NLB security group rule allowing tcp/80 only from that NAT EIP. As a result, the internet-facing NLB was not reachable from my browser. Removing the source range made the UI accessible.

Evidence from the environment

The NLB security group had only an inbound rule equivalent to:

Rule: tcp/80 from <workshop-nat-gateway-eip>/32
Tags: service.k8s.aws/stack = ui/ui-nlb

The same IP was the NAT Gateway EIP created by the workshop stack:

Elastic IP: <workshop-nat-gateway-eip>
Name tag: eksctl-eks-workshop-cluster/NATIP
CloudFormation logical ID: NATIP

CloudTrail showed the rule was created by AWS Load Balancer Controller:

eventName: AuthorizeSecurityGroupIngress
userAgent: elbv2.k8s.aws/v2.17.1
assumed role: eks-workshop-alb-controller-*
cidrIpv4: <workshop-nat-gateway-eip>/32

Testing from my browser/local public IP failed until the source-range annotation was removed.

Expected behavior

The lab says the NLB DNS can be used to access the application from outside the Kubernetes cluster. With the current auto-detected CIDR behavior, the generated NLB may only allow requests that originate from the workshop VPC NAT Gateway, which blocks normal browser access from the participant machine.

Why this is confusing

The NLB is internet-facing and placed in public subnets, but the source range can be set to the NAT Gateway EIP. That NAT is only in the path for workloads/environments inside private subnets going outbound. It is not in the path for a participant browser accessing the public NLB from the internet.

Possible fixes

  • Clarify that SOURCE_IP_ADDRESS must be set to the participant browser/client public IP before creating the NLB.
  • Avoid using an auto-detected NAT Gateway EIP for browser-facing NLB labs.
  • Add troubleshooting steps showing how to inspect the generated NLB security group and update/remove INBOUND_CIDRS.
  • Consider printing a warning when SOURCE_IP_ADDRESS=auto is resolved from inside the workshop environment, since it may represent the NAT Gateway rather than the browser client.

Workshop area

  • manifests/modules/exposing/load-balancer/nlb/nlb.yaml
  • website/docs/fundamentals/exposing/loadbalancer/adding-lb.md
  • hack/lib/resolve-source-ip.sh
  • docs/authoring_content.md

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    Status
    No status

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions