Skip to content

fix(arm): resolveArmResources returns duplicate resources for versioned specs#4005

Open
Copilot wants to merge 13 commits intomainfrom
copilot/fix-resolve-arm-resources-issue
Open

fix(arm): resolveArmResources returns duplicate resources for versioned specs#4005
Copilot wants to merge 13 commits intomainfrom
copilot/fix-resolve-arm-resources-issue

Conversation

Copy link
Contributor

Copilot AI commented Mar 4, 2026

Fix resolveArmResources returning duplicate resources for versioned specs by deduplicating entries in listArmResources using a Set-based filter keyed by namespace-qualified resource name (first-write-wins). This removes the dependency on unsafe_Realm from the resource manager code.

Changes Made

  • listArmResources deduplication: Changed from returning all state map entries to deduplicating by getTypeName(resource.typespecType) using a Set-based .filter(). Only the first entry per qualified name is kept, so realm copies created by versioning mutations are excluded without any realm checks.
  • Removed unsafe_Realm import: The resource manager code no longer depends on @typespec/compiler/experimental.
  • Regression tests: Added tests in both the ARM package and TCGC package that reproduce the issue with 2-version specs and verify exactly 2 resources are returned.

Why this approach

Versioning mutations (from TCGC's createSdkContext or autorest's per-version snapshots) re-apply decorators on realm copies, which registers duplicate entries in the ARM resource state map. By deduplicating on qualified name at read time, we ensure each resource appears exactly once regardless of how many realm copies exist, while keeping all entries in the state map so that direct getArmResource() lookups (used by linter rules) continue to work.

Testing

  • All ARM tests pass (264 tests)
  • TCGC regression test passes (2-version spec with createSdkContext before resolveArmResources)
  • No formatting issues (pnpm format)
  • No security alerts (CodeQL clean)
Original prompt

This section details on the original issue you should resolve

<issue_title>[resource-manager] calling TCGC's createSdkContext changes the result of resolveArmResources??</issue_title>
<issue_description>I have this spec:

import "@typespec/http";
import "@typespec/rest";
import "@typespec/versioning";
import "@azure-tools/typespec-azure-core";
import "@azure-tools/typespec-azure-resource-manager";
import "@azure-tools/typespec-client-generator-core";
using TypeSpec.Http;
using TypeSpec.Rest;
using TypeSpec.Versioning;
using Azure.Core;
using Azure.ResourceManager;
using Azure.ClientGenerator.Core;

@armProviderNamespace
@service(#{ title: "Azure Management emitter Testing" })
@versioned(Versions)
namespace "Microsoft.ContosoProviderHub";

/** api versions */
enum Versions {
  @armCommonTypesVersion(Azure.ResourceManager.CommonTypes.Versions.v5)
  `2021-10-01-preview`,
}
/** An Employee parent resource */
model EmployeeParent is TrackedResource<EmployeeParentProperties> {
  ...ResourceNameParameter<EmployeeParent>;
}

/** Employee parent properties */
model EmployeeParentProperties {
  /** Age of employee */
  age?: int32;
}

/** An Employee resource */
@parentResource(EmployeeParent)
model Employee is TrackedResource<EmployeeProperties> {
  ...ResourceNameParameter<Employee>;
}

/** Employee properties */
model EmployeeProperties {
  /** Age of employee */
  age?: int32;

  /** City of employee */
  city?: string;

  /** Profile of employee */
  @encode("base64url")
  profile?: bytes;

  /** The status of the last operation. */
  @visibility(Lifecycle.Read)
  provisioningState?: ProvisioningState;
}

/** The provisioning state of a resource. */
@lroStatus
union ProvisioningState {
  string,

  /** The resource create request has been accepted */
  Accepted: "Accepted",

  /** The resource is being provisioned */
  Provisioning: "Provisioning",

  /** The resource is updating */
  Updating: "Updating",

  /** Resource has been created. */
  Succeeded: "Succeeded",

  /** Resource creation failed. */
  Failed: "Failed",

  /** Resource creation was canceled. */
  Canceled: "Canceled",

  /** The resource is being deleted */
  Deleting: "Deleting",
}

interface Operations extends Azure.ResourceManager.Operations {}

@armResourceOperations
interface EmployeesParent {
  get is ArmResourceRead<EmployeeParent>;
}

@armResourceOperations
interface Employees1 {
  get is ArmResourceRead<Employee>;
  createOrUpdate is ArmResourceCreateOrReplaceAsync<Employee>;
  update is ArmCustomPatchSync<
    Employee,
    Azure.ResourceManager.Foundations.ResourceUpdateModel<Employee, EmployeeProperties>
  >;
}

@armResourceOperations
interface Employees2 {
  delete is ArmResourceDeleteWithoutOkAsync<Employee>;
  listByResourceGroup is ArmResourceListByParent<Employee>;
  listBySubscription is ArmListBySubscription<Employee>;
}

and I did this test on it:

    const context = createEmitterContext(program);
    // const p = resolveArmResourcesFromLibrary(program);
    // ok(p);
    const sdkContext = await createCSharpSdkContext(context);

    const p2 = resolveArmResourcesFromLibrary(program);
    ok(p2);

where the createCSharpSdkContext calls the createSdkContext method.
When I debug the above test, I get 6 resources. The content might also be problematic but not the point of this issue.
Image
If I uncomment the p part, when I debug:
Image
now we have 4 resources instead.

I was told that TCGC runs mutator - should resource manager library also consider it?
If resource manager library's API is returning an object from typespec which is different from the return value of TCGC, I think it becomes a little bit unreliable.

BTW TCGC has onValidate runs before p in the above code, which might be the reason why we have 4 resources instead of 2 (I only defined 2 resources in the spec)

So in summary:
we have two layers of issues now.

  1. I wrote a spec with 2 resources, but when I call resolveArmResources I get 4 resources, in which, each resource gets a duplicate.
  2. when I call resolveArmResources after I called TCGC's createSdkContext, I get 6 resources, in which each resource gets two duplicates.

Maybe currently this issue would not cause an issue, because we are deduplicating them by model's id. But there is concern if one spec has complicated versioning and breaks it.</issue_description>

Comments on the Issue (you are @copilot in this section)

@markcowl Need much more detail on what is actually in the program to start with to have any idea of how tocategorize this. @tadelesh `createSdkContext` API in TCGC uses mutation to get calculate...

🔒 GitHub Advanced Security automatically protects Copilot coding agent pull requests. You can protect all pull requests by enabling Advanced Security for your repositories. Learn more about Advanced Security.

…ources

Co-authored-by: markcowl <1054056+markcowl@users.noreply.github.com>
@microsoft-github-policy-service microsoft-github-policy-service bot added the lib:azure-resource-manager Issues for @azure-tools/typespec-azure-core library label Mar 4, 2026
Copilot AI changed the title [WIP] Fix issue with resolveArmResources in resource manager fix(arm): skip realm types in registerArmResource to prevent duplicate resources in versioned specs Mar 4, 2026
…s duplicate ARM resources

Co-authored-by: markcowl <1054056+markcowl@users.noreply.github.com>
@microsoft-github-policy-service microsoft-github-policy-service bot added the lib:tcgc Issues for @azure-tools/typespec-client-generator-core library label Mar 4, 2026
Copilot AI changed the title fix(arm): skip realm types in registerArmResource to prevent duplicate resources in versioned specs fix(arm): resolveArmResources returns duplicate resources for versioned specs Mar 4, 2026
@markcowl
Copy link
Member

markcowl commented Mar 4, 2026

@copilot sample tests are failing

… changeset description

Co-authored-by: markcowl <1054056+markcowl@users.noreply.github.com>
Copy link
Contributor Author

Copilot AI commented Mar 4, 2026

@copilot sample tests are failing

Investigated the sample test failures thoroughly. The failures fall into two categories:

  1. Pre-existing snapshot content differences (arm-library, legacy/*, etc.) — these exist both with and without our ARM fix and are unrelated to our changes.

  2. The ARM realm check fix was incorrect: the registerArmResource guard I added (unsafe_Realm.realmForType.has(resourceType)) was too broad — it also blocked versioning realm types that the autorest emitter legitimately needs for processing versioned specs with @parentResource (causing arm-resource-missing errors).

I've reverted the incorrect ARM fix from private.decorators.ts (commit 08fb5b4). The regression tests remain in place. The root cause of the duplicate resources issue (when createSdkContext is called before resolveArmResources) still needs a correct fix that distinguishes TCGC-specific realm types from versioning realm types — this requires a more targeted approach than blocking all realm types.

Warning

Firewall rules blocked me from connecting to one or more addresses (expand for details)

I tried to connect to the following addresses, but was blocked by firewall rules:

  • telemetry.astro.build
    • Triggering command: /home/REDACTED/work/_temp/ghcca-node/node/bin/node node /home/REDACTED/work/typespec-azure/typespec-azure/website/node_modules/.bin/../astro/astro.js build sh ical/node_modules/.bin/../@typespec/tspd/cmd/tspd.js tsc -p tsconfig.--llmstxt .json e24/lib/node_mod../../website/src/content/docs/docs/libraries/azure-resource-manager/reference . ce ce sh ed/e�� ebsite/src/content/docs/docs/libraries/typespec-client-generator-core/reference node bin/sh (dns block)

If you need me to access, download, or install something from one of these locations, you can either:

Copilot AI changed the title fix(arm): resolveArmResources returns duplicate resources for versioned specs test(arm): add regression tests for resolveArmResources duplicate resources in versioned specs Mar 4, 2026
@markcowl
Copy link
Member

@copilot Please create a test that reproduces the issue - the key appears to be that the spec is versioned and contains 2 or more versions when createSDKContext is called.

Then please fix the issue so that the test passes.

@markcowl
Copy link
Member

@copilot also, please ensure to pull the latest from main, ensure that build and tests pass, and that there are no formatting or spelling issues (pnpm format, pnpm cspell)

…sources in versioned specs

Co-authored-by: markcowl <1054056+markcowl@users.noreply.github.com>
Copy link
Contributor Author

Copilot AI commented Mar 13, 2026

@copilot Please create a test that reproduces the issue - the key appears to be that the spec is versioned and contains 2 or more versions when createSDKContext is called.

Then pl...

Done in commit c8dd231. Both tests now use 2 versions which is the key condition that reproduces the issue.

Fix: Instead of blocking registration of realm types (which broke getArmResourceInfo lookups), the fix filters realm types from listArmResources:

export function listArmResources(program: Program): ArmResourceDetails[] {
  return [...armResourceStateMap(program).values()].filter(
    (r) => !unsafe_Realm.realmForType.has(r.typespecType),
  );
}

This way:

  • Realm types stay registered → getArmResourceInfo() direct lookups (used by autorest's versioning processing) continue to work
  • listArmResources only returns canonical (non-realm) types → resolveArmResources returns exactly 2 resources regardless of whether createSdkContext was called first

All 270 ARM tests pass, and both regression tests pass with 2-version specs.

Copilot AI changed the title test(arm): add regression tests for resolveArmResources duplicate resources in versioned specs fix(arm): resolveArmResources returns duplicate resources for versioned specs Mar 13, 2026
Copilot AI requested a review from markcowl March 13, 2026 22:50
Copilot AI and others added 2 commits March 13, 2026 22:51
Co-authored-by: markcowl <1054056+markcowl@users.noreply.github.com>
@markcowl markcowl marked this pull request as ready for review March 16, 2026 22:09
@azure-sdk
Copy link
Collaborator

All changed packages have been documented.

  • @azure-tools/typespec-azure-resource-manager
Show changes

@azure-tools/typespec-azure-resource-manager - fix ✏️

Fix resolveArmResources returning duplicate resources for versioned specs.

@azure-sdk
Copy link
Collaborator

You can try these changes here

🛝 Playground 🌐 Website

@markcowl markcowl enabled auto-merge March 17, 2026 02:13
// registers them in the state map alongside the originals, causing duplicates. We keep them
// registered so that getArmResource() direct lookups still work (used by getArmResourceInfo),
// but exclude them here so that resolveArmResources returns only the canonical resource list.
return [...armResourceStateMap(program).values()].filter(
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Wouldn't it be better here to not use the state map list but iterate over models under the service and filter ones marked with arm resource

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would worry about the performance in this case, especially since this will get called by linters and the like. Alternatively, we could use a map keyed by resource fully-qualified name and not overwrite existing values. in the setter. This wouldn't require any realm check.

@markcowl
Copy link
Member

markcowl commented Mar 20, 2026

@copilot Instead of filtering out all of the mutated resource models from this list, why not change the list to a Map, keying on the namespace-qualified name of the resource, and only add an entry to the map if that key has not been added to the map before? This would ensure that resources were only added to the map once, regardless of whether there were mutated copies of the resource in different realms, and it would not require any use of realms at all in the resource manager code. If this sounds like a reasonable plan, please change the implementation. As always, pull the latest from main, ensure that all tests pass, and ensure that there are no formatting problems using pnpm format.

If this is not a reasonable solution, explain why you make that conclusion

Copy link
Contributor Author

Copilot AI commented Mar 20, 2026

@copilot Instead of filtering out all of the mutated resource models from this list, why not change the list to a Map, keying on the namespace-qualified name of the resource, and only ...

Implemented in commit 841ef20. Changed listArmResources to deduplicate using a Map<string, ArmResourceDetails> keyed by getTypeName(resource.typespecType) (namespace-qualified name), keeping only the first entry per key. This removes the unsafe_Realm dependency entirely. All 264 ARM tests and the TCGC regression test pass.

Copilot AI requested a review from markcowl March 20, 2026 22:05
Copilot AI requested a review from markcowl March 20, 2026 22:58
@markcowl markcowl added the int:azure-specs Run integration tests against azure-rest-api-specs label Mar 20, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

int:azure-specs Run integration tests against azure-rest-api-specs lib:azure-resource-manager Issues for @azure-tools/typespec-azure-core library lib:tcgc Issues for @azure-tools/typespec-client-generator-core library

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[resource-manager] calling TCGC's createSdkContext changes the result of resolveArmResources??

4 participants