Skip to content

fix: merge or refuse an entity's conflicting definitions rather than dropping one - #681

Merged
sini merged 5 commits into
mainfrom
fix/instance-collection-merge
Sep 16, 2026
Merged

sini merged 5 commits into
mainfrom
fix/instance-collection-merge

Conversation

@sini

@sini sini commented Sep 16, 2026 •

Copy link
Copy Markdown
Collaborator

#678 (comment)

Summary

Merge:

  • Merges an entity's list-valued keys instead of overwriting them, as den.hosts' freeform type merged with lib.recursiveUpdate, which treats a list as an opaque leaf, so two files each writing den.hosts.<system>.<name>.includes kept one list and dropped the other with no conflict warning.
  • Concatenates rather than picking a winner, because that is the module system's own rule for a list-valued option (listOf merges by concatLists) and den's rule at the aspect tier (aspectContentType's deepMerge), so the entity registry now agrees with both instead of adding a third behaviour.
  • Applies to every list-valued key an entity carries, since includes, excludes and classes accumulate everywhere else they appear.
  • Concatenates in declaration order, as defs reach this merge in reverse declaration order, which is measured rather than assumed and is the same ordering that makes the scalar arm read as first-declaration-wins.

Conflicts:

  • Refuses two definitions of a key that is neither a mergeable attribute set nor a list, as they previously took the first and dropped the other, and a type mismatch such as a list against a string resolved the same silent way.
  • Refuses when either side is a plain scalar and the values differ, which covers both arms. Requiring both sides to be scalars leaves the mismatch silent, which is what an earlier form of the predicate did.
  • Excludes functions deliberately, since == on two functions is always false and comparing them would reject two identical definitions of a class module or of instantiate.
  • Names a value in the message only where rendering it is safe, because builtins.toJSON throws on a derivation or a function and either can be one side of a conflict.

Why it went unnoticed:

  • An attrset key under the same two definitions merged normally, so the collection read as having no effect rather than as a failed merge. A users.alice and users.bob pair across two files both survive, which is what localised the defect to list leaves rather than to the collection keys or the registry's recursion.
  • Instance-level collections were read by nothing until recently, so there was no way to observe which list survived.
  • Both arms are PRE-EXISTING rather than regressions, measured against a tree predating the instance collections being read at all, where the scalar conflict and the type mismatch resolve identically. Making the collections meaningful changed observability, not behaviour, so a list key on an entity has been losing definitions since the gen-schema entity port.

gen hub

Bumps templates/ci/flake.lock's gen pin from 0b6fbd8d to 0004f3c9 over the two chore: commits, which moves gen-schema f8e0e171 to 48cfb992 and leaves gen-merge held at 08fcdd1 on the flake path, so nothing above is affected.

  • Fixes a separate report, a den.schema.user option declared without default = { } failing with gen-merge: the option ... is used but not defined. The first of the two bumps deliberately did NOT fix it, and said so, because the cause was never the hub rev.
  • The cause is entry-path divergence. nix/lib/schema.nix dispatches on inputs ? gen: only templates/ci declares that input, so den's own suite takes the FLAKE path while every other consumer takes the builtins.fetchTarball fallback. The fallback enters the hub standalone, where no follows exist and each member resolves its deps from its own lock, so one hub rev yielded two different closures — gen-merge 08fcdd1 (guarded) on the flake path and f7e3afb3 (pre-guard) standalone.
  • gen-schema 48cfb992 carries 2a33f81 in both of its own locks, which is what makes the two paths agree rather than the hub pin moving.
  • Also drops the provider test fixture's gen-schema input, which was referenced nowhere and pinned 4bd0f6eb, a rev predating gen-merge entirely. It put a second gen-schema node in the CI lock, so one lock carried two revisions of one library — the incoherence the hub indirection exists to prevent, in the tree that documents it.

Behaviour changes

  • A list-valued key on an entity now accumulates across definitions where one definition previously overwrote the other. A configuration that wrote the same key in two files and relied on one winning will start seeing both.

  • Two definitions of a non-mergeable key now refuse instead of silently resolving to the first. A configuration that wrote a key twice with different values, or with mismatched types, will start failing where it previously picked one. Definitions that agree are unaffected.

  • A consumer that declares no gen input now resolves gen-merge 2a33f81 where it previously resolved f7e3afb3. That is the fix for the report above, and it is the only closure change on the fallback path.

Validation

  • nix develop -c just ci: 1221/1221, exit 0, zero failures and zero errors, read off stderr with the collected count agreeing with the numerator.

  • Falsified the three defect cells against the unfixed merge with the tests held in place, so the red arm measures the fix rather than the cells' absence. Both control cells stay green in that arm, which is what makes them controls rather than padding.

  • Pinned declaration order in its own cell, as concatenating in either direction makes both entries present and so passes the other cells.

  • Reproduced the report before changing anything, and measured the surviving entry rather than inferring it: the first definition won, not the last.

  • Measured both arms against a pre-existing tree rather than reasoning about blame, which is what established these as latent defects rather than regressions.

  • Verified the hub bump against the reporting user's real configuration rather than a reduction, with a live control: home-manager.users.<u>.programs.helix.ignores yields its list at exit 0 on this branch, and the same attribute at the previous pin, same instrument, exits 1 with gen-merge: the option ... is used but not defined.

  • nix flake check on minimal, default and noflake, none of which declares a gen input, so all three exercise the fetched fallback that consumers take rather than the flake path the suite measures.

Two files each writing `den.hosts.<system>.<name>.includes` kept one list and
dropped the other, with no conflict warning. Reported by theutz on #678.

`den.hosts`' freeform type merged with `lib.recursiveUpdate`, which treats a
list as an opaque leaf, so one definition overwrote the other. An attrset key
under the same two definitions merged normally, which is what made the
collection look like it had no effect rather than like a merge failure.

Concatenation is the module system's own rule for a list-valued option
(`listOf` merges by `concatLists`) and den's rule at the aspect tier
(`aspectContentType`'s `deepMerge`), so this makes the entity registry agree
with both rather than add a third behaviour. `includes`, `excludes` and
`classes` are the list-valued keys an entity carries, and all three accumulate
everywhere else they appear.

Latent until instance-level collections started being read, since before that
nothing consumed them and there was no way to observe which list survived.

Concatenates `bv ++ a.<k>` rather than the other way round because defs reach
this merge in reverse declaration order, measured rather than assumed. It is
the same ordering that makes the scalar arm read as first-declaration-wins,
and include order decides which aspect's content wins a conflict, so the
direction is pinned by its own cell.

Validation: `just ci` 1217/1217, exit 0, zero failures and zero errors. The
three defect cells falsified against the unfixed merge with the tests held in
place, where both control cells stay green in that arm, so they are controls
rather than padding.
@sini
sini requested a review from theutz September 16, 2026 12:50
@sini
sini requested a review from vic as a code owner September 16, 2026 12:50
@github-actions github-actions Bot added the allow-ci allow all CI integration tests label Sep 16, 2026
The same silent resolution as the list overwrite, one arm over. Two
definitions of a key that is neither a mergeable attribute set nor a list took
the first and dropped the other, and a TYPE MISMATCH resolved the same way: a
list in one file against a string in another silently kept the list.

Pre-existing, and measured as such against a tree predating the instance
collections being read at all, where both arms resolve identically. It is also
weaker than the module system, which errors on a DECLARED option with
conflicting definitions. An entity's freeform keys bypassed that.

Refuses when either side is a plain scalar and the values differ, which covers
two scalars that disagree and a mismatch such as a list against a string. An
earlier form required BOTH sides to be scalars and left the mismatch silent,
which its own cell caught.

Functions are excluded deliberately: `==` on two functions is always false, so
comparing them would reject two identical definitions of a class module or of
`instantiate`. A control cell pins that.

The message names a value only where rendering it is safe. `builtins.toJSON`
throws on a derivation or a function, and either can be one side of a
conflict, so a message that always rendered both would fail while reporting a
failure.

Validation: `just ci` 1221/1221, exit 0, zero failures and zero errors. Both
refusal cells falsified against the unfixed predicate with the tests held in
place, where the two controls stay green, so they are controls rather than
padding.
@sini sini changed the title fix: merge an entity's list-valued keys instead of overwriting them fix: merge or refuse an entity's conflicting definitions rather than dropping one Sep 16, 2026
@sini
sini enabled auto-merge (squash) September 16, 2026 13:49
@sini
sini disabled auto-merge September 16, 2026 13:54
Moves gen-schema f8e0e171 to 056ee9b513. gen-merge stays held at 08fcdd1efb,
so discussion #672's fix is unaffected.

Does NOT fix the `option ... is used but not defined` regression a user hit on
a defaultless schema submodule option, and that is measured rather than
assumed: their real configuration still fails against this bump. The cause is
entry-path dependent and this pin does not touch it. The hub's ci lock pins
gen-merge 08fcdd1efb while gen-schema's OWN ci lock pins f7e3afb3, which
predates the `hasEmptyValue` guard. On the flake path the hub's `follows`
override that and 08fcdd1efb wins, which is why this suite is green. On the
standalone path there are no follows, each member resolves its own deps from
its own ci lock, and gen-schema's pin wins — so one hub rev yields two
different closures depending on how it is entered. den's fallback takes the
standalone path, so every consumer that does not declare `gen` gets the older
gen-merge.

Validation: `just ci` 1221/1221, exit 0, zero failures and zero errors.
`nix flake check` on `minimal` and `default`, both of which declare no `gen`
input and so exercise the fetched fallback rather than the flake input —
the path a previous bump validated only by accident.
Moves gen-schema 056ee9b513 to 48cfb99289. gen-merge stays held at 08fcdd1efb
on the flake path, so discussion #672's fix is unaffected.

This one DOES fix the `option ... is used but not defined` regression on a
defaultless schema submodule option, which the previous bump did not. The
cause was never the hub rev: gen-schema's own locks pinned gen-merge
f7e3afb3, which predates the `hasEmptyValue` guard, and den's fallback enters
the hub standalone where no `follows` exist and each member resolves from its
own lock. gen-schema 48cfb99289 carries 2a33f81444 in BOTH of its locks, so
the two entry paths now agree.

Validation, with a live control because a green suite here measures only the
flake path:

- The reporting user's real configuration, evaluated against this checkout:
  `home-manager.users.<u>.programs.helix.ignores` yields its list, exit 0.
  Same attribute at the previous pin, same instrument: exit 1,
  `gen-merge: the option ... is used but not defined`.
- `just ci` 1221/1221, exit 0, zero failures and zero errors.
- `nix flake check` on `minimal` and `default`, neither of which declares a
  `gen` input, so both exercise the fetched fallback that consumers take.
The `provider` test fixture declared `gen-schema` directly and referenced it
nowhere — measured: its whole tree is `flake.nix`, `flake.lock` and
`modules/den.nix`, and `gen-schema` appears only in the two input lines this
removes.

It was not inert. A direct pin put a SECOND gen-schema node in the CI lock at
4bd0f6eb17, a rev predating gen-merge entirely, so one lock carried two
revisions of one library — the exact incoherence the hub indirection in
`nix/lib/schema.nix` exists to prevent, sitting in the tree that documents it.

Validation: `just ci` 1221/1221, exit 0, zero failures and zero errors. The
fixture is exercised by the namespace-provider and external-namespace suites,
so the suite measures the removal rather than just tolerating it.

@theutz theutz left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice! Can confirm this fixes the problem on my local.

# first-declaration-wins.
bv ++ a.${bk}
else if
# Either side a plain scalar means the two are not both mergeable

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a good catch!

@sini
sini enabled auto-merge (squash) September 16, 2026 16:22
@sini
sini merged commit 20d1e76 into main Sep 16, 2026
50 of 56 checks passed
@sini
sini deleted the fix/instance-collection-merge branch September 16, 2026 16:30
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

allow-ci allow all CI integration tests

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants