Skip to content

feat: make AWS::LanguageExtensions local processing opt-in via --language-extensions flag#9033

Merged
bnusunny merged 33 commits into
developfrom
le-disable-and-backcompat-tests
May 21, 2026
Merged

feat: make AWS::LanguageExtensions local processing opt-in via --language-extensions flag#9033
bnusunny merged 33 commits into
developfrom
le-disable-and-backcompat-tests

Conversation

@bnusunny
Copy link
Copy Markdown
Contributor

@bnusunny bnusunny commented May 20, 2026

Summary

  • Make local AWS::LanguageExtensions processing opt-in (off by default), restoring pre-1.160.0 behavior for templates that declare the transform but don't need local expansion
  • Add --language-extensions / --no-language-extensions CLI flag to all 8 affected commands: build, package, deploy, sync, validate, local invoke, local start-api, local start-lambda
  • Support SAM_CLI_ENABLE_LANGUAGE_EXTENSIONS=1 env var as fallback (flag wins)
  • Support samconfig.toml persistence via language_extensions = true

Motivation

SAM CLI 1.160.0 enabled local processing of AWS::LanguageExtensions implicitly when the transform was declared. This caused regressions (#9004, #9005, #9027, #9029) for templates that declared the transform for non-Fn::ForEach reasons or as a forward-compat placeholder. Making it opt-in lets customers adopt at their own pace.

Changes

  1. expand_language_extensions() now requires enabled: bool as a keyword-only arg — unwired callers fail with TypeError
  2. resolve_language_extensions_enabled(flag_value: Optional[bool]) -> bool resolver (flag → env var → False)
  3. @language_extensions_option Click decorator shared across all commands
  4. SamLocalStackProvider.get_stacks(..., language_extensions_enabled=False) kwarg threaded through recursion
  5. Each command context (Build, Package, Deploy, Sync, Validate, InvokeContext) wired end-to-end
  6. Template class in artifact_exporter wired for nested-stack recursion
  7. Inspection-path callers (sam list, companion stack, image-repo validation) explicitly pass False
  8. Telemetry regression tests lock in: event fires only when opt-in AND template uses LE
  9. Integration tests updated to pass --language-extensions for ForEach templates
  10. Documentation updated with opt-in instructions

Test plan

  • 9237 unit tests pass (pytest tests/unit/ -q -W ignore::pytest.PytestUnknownMarkWarning)
  • --language-extensions parameter registered on all 8 commands (verified programmatically)
  • make format clean
  • make lint shows zero errors
  • Integration test test_build_without_flag_does_not_expand_foreach verifies opt-out behavior
  • Existing ForEach integration tests pass with --language-extensions flag

bnusunny added 19 commits May 19, 2026 20:43
Add language_extensions_enabled kwarg to SamLocalStackProvider.get_stacks
with default=False to match pre-1.160.0 behavior. Forward to
expand_language_extensions and recursive get_stacks calls for nested
stacks.
…alidator

Wire the --language-extensions flag from validate command through to
SamTemplateValidator and expand_language_extensions. Validator takes a
resolved bool (not Optional) since resolution happens at command boundary.

Add language_extensions parameter to __init__, forward enabled kwarg to
expand_language_extensions call, and update all test call sites.
@bnusunny bnusunny requested a review from a team as a code owner May 20, 2026 03:54
@github-actions github-actions Bot added area/package sam package command area/deploy sam deploy command area/validate sam validate command area/build sam build command area/local/invoke sam local invoke command area/sync sam sync command area/schema JSON schema file pr/internal labels May 20, 2026
Copy link
Copy Markdown
Collaborator

@aws-sam-cli-bot aws-sam-cli-bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review Results

Reviewed: 9ffa5ed..0d96fac
Files: 60
Comments: 1

Comment thread tests/integration/buildcmd/test_build_cmd_language_extensions.py
Copy link
Copy Markdown
Collaborator

@aws-sam-cli-bot aws-sam-cli-bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review Results

Reviewed: 9ffa5ed..bdcd614
Files: 66
Comments: 1


Comments on lines outside the diff:

[samcli/commands/deploy/command.py:324] [BUG] The sam deploy command's inner PackageContext(...) invocation is not threaded with the new language_extensions flag, while the surrounding GuidedContext and DeployContext calls are. As a result, when a user runs sam deploy --language-extensions against a template that needs LE expansion, the package phase (which performs the artifact upload and walks/expands child stacks via artifact_exporter.Template) will silently run with language_extensions_enabled=False, while the deploy phase runs with it enabled.

Concrete consequences:

  • Artifacts referenced inside Fn::ForEach blocks (including in nested stacks reached through artifact_exporter.Template's recursion) will not be uploaded to S3/ECR, because expand_language_extensions(...) returns the passthrough result and the dynamic artifact properties needed by Template.export() are never computed.
  • The deployed CloudFormation template may reference local code paths or unresolved artifacts that the package step skipped.
  • sam sync (which also chains Build → Package → Deploy) correctly passes language_extensions=language_extensions into both PackageContext and DeployContext (see samcli/commands/sync/command.py lines 373 and 410 in this PR), so the deploy command is the inconsistent one.

Fix: pass language_extensions=language_extensions to the PackageContext(...) call at this site, mirroring the sync command's wiring:

with PackageContext(
   template_file=template_file,
   ...
   parameter_overrides=context_param_overrides,
   language_extensions=language_extensions,
) as package_context:
   package_context.run()

Without this change, the new --language-extensions flag is effectively half-applied for sam deploy, which can produce broken deployments for the very templates this flag was added to support.

Copy link
Copy Markdown
Collaborator

@aws-sam-cli-bot aws-sam-cli-bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review Results

Reviewed: 9ffa5ed..ed0d4f9
Files: 66
Comments: 1


Comments on lines outside the diff:

[samcli/lib/sync/infra_sync_executor.py:609] [BUG] With the new opt-in semantics, _detect_foreach_code_changes can crash on templates that declare AWS::LanguageExtensions but where the user did not pass --language-extensions. The early-return guard only short-circuits when the template has no LE transform at all:

if not check_using_language_extension(current_template):
   return
...
current_expanded = expand_language_extensions(
   current_template,
   parameter_values=current_params,
   enabled=self._sync_context.language_extensions_enabled,
).expanded_template

When the transform is declared but enabled=False, expand_language_extensions now returns a passthrough (per the new if not enabled branch in sam_integration.py), so current_expanded still contains the unexpanded Fn::ForEach::* keys whose values are lists, not dicts. The subsequent loop then does:

for resource_id, resource_dict in current_resources.items():
   if resource_id in original_regular_keys:  # Fn::ForEach key is excluded from this set
       continue
   resource_type = resource_dict.get("Type")  # AttributeError: 'list' object has no attribute 'get'

This is the exact regression scenario the PR aims to support (template declares the transform, user opts out), and the call site in _auto_skip_infra_sync has no surrounding try/except, so the AttributeError propagates and breaks sam sync's auto-skip-infra path.

Add an early return for the disabled case, e.g.:

if not self._sync_context.language_extensions_enabled:
   return
if not check_using_language_extension(current_template):
   return

Or guard the loop body to skip is_foreach_key(resource_id) entries when expansion was a passthrough.

@bnusunny
Copy link
Copy Markdown
Contributor Author

Re: PackageContext not threaded in deploy command (review 2026-05-20T17:51)

Fixed in ed0d4f9language_extensions=language_extensions is now passed to the PackageContext(...) call inside do_cli, matching the sync command's wiring.

@bnusunny
Copy link
Copy Markdown
Contributor Author

Re: _detect_foreach_code_changes crash when LE disabled (review 2026-05-20T18:04)

Fixed in 9f5934b — added early return when language_extensions_enabled is False, merged with the existing check_using_language_extension guard into a single condition.

valerena
valerena previously approved these changes May 20, 2026
Copy link
Copy Markdown
Contributor

@valerena valerena left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just a couple of questions, but it looks good.

Comment thread samcli/lib/cfn_language_extensions/sam_integration.py
Comment thread samcli/lib/cli_validation/image_repository_validation.py
Comment thread tests/unit/commands/deploy/test_command.py
Copy link
Copy Markdown
Collaborator

@aws-sam-cli-bot aws-sam-cli-bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review Results

Reviewed: 9ffa5ed..d0f7f41
Files: 66
Comments: 1

Comment thread samcli/lib/cli_validation/image_repository_validation.py
Comment thread samcli/commands/build/build_context.py
Comment thread samcli/commands/_utils/options.py
Comment thread samcli/commands/deploy/deploy_context.py
Comment thread samcli/commands/local/cli_common/invoke_context.py
Comment thread samcli/lib/cfn_language_extensions/sam_integration.py
Copy link
Copy Markdown
Collaborator

@aws-sam-cli-bot aws-sam-cli-bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review Results

Reviewed: 78c43a8..8ceafc3
Files: 66
Comments: 1

Comment thread samcli/lib/bootstrap/companion_stack/companion_stack_manager.py
@bnusunny bnusunny enabled auto-merge May 20, 2026 23:17
@bnusunny bnusunny added this pull request to the merge queue May 20, 2026
@github-merge-queue github-merge-queue Bot removed this pull request from the merge queue due to failed status checks May 20, 2026
@bnusunny bnusunny enabled auto-merge May 21, 2026 00:49
@bnusunny bnusunny added this pull request to the merge queue May 21, 2026
Merged via the queue into develop with commit fe16ab0 May 21, 2026
53 checks passed
bnusunny added a commit that referenced this pull request May 21, 2026
After #9033, language extensions are opt-in. The two ForEach zipfile
tests added by this PR omitted the flag, causing SAM transform plugins
to choke on unexpanded Fn::ForEach blocks.
bnusunny added a commit that referenced this pull request May 21, 2026
PR #9033 made AWS::LanguageExtensions local processing opt-in. Two API
changes broke three tests on this branch after the merge from develop:

- expand_language_extensions() now requires a keyword-only `enabled` arg
- PackageContext reads self._language_extensions_enabled, set in __init__

Pass enabled=True to expand_language_extensions in the two artifact-exporter
tests, and set _language_extensions_enabled on the PackageContext stub that
bypasses __init__ via __new__. All three tests exercise templates with
Transform: AWS::LanguageExtensions, so enabling LE is the intended behavior.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area/build sam build command area/deploy sam deploy command area/local/invoke sam local invoke command area/package sam package command area/schema JSON schema file area/sync sam sync command area/validate sam validate command pr/internal

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants