Skip to content

Request local Modules by folder with path: in modules.yaml - #115

Merged
Jiu-xiao merged 10 commits into
devfrom
feat/local-modules-20261109
Nov 10, 2026
Merged

Jiu-xiao merged 10 commits into
devfrom
feat/local-modules-20261109

Conversation

@Jiu-xiao

@Jiu-xiao Jiu-xiao commented Nov 10, 2026 •

Copy link
Copy Markdown
Member

A Module used by one BSP only can live in a folder and be requested from modules.yaml with path:, without a Git repository or a source of its own.

xrobot: 1.0.2
modules:
  - xrobot-org/BlinkLED@same-or-dev
  - path: Modules/local/Heartbeat

Changes

  • modules.yaml accepts - path: <folder>, relative to the BSP root or absolute. The Module id is local/<folder name>, and the folder holds the primary header of the same name. The last segment of the path must be the folder's real name (case and symlinks included). A path that names the BSP root, Modules/ or a folder containing the BSP, or holds a semicolon or control character, is rejected.
  • xrobot.lock gains an optional top-level local: section that copies the path of each local Module from modules.yaml. --frozen fails when it differs; a plain setup rewrites it. A project without path: items writes no such section.
  • xrobot module add --path <folder> checks the folder and its primary header and adds the entry; a relative path is taken from the current directory first, then from the BSP root, and written relative to the BSP root. xrobot module remove local/<Name> removes it.
  • A local Module is used as its folder currently holds it. One inside the BSP is pinned by the BSP's commits; setup fails when the BSP's .gitignore ignores one of its untracked source, header or CMake files and names the line to add (such as !/Modules/local/ after /Modules/*/), and warns when the folder has its own .git. One outside the BSP only produces a warning.
  • The dependencies of a local Module count as requests of modules.yaml: they follow the BSP branch and take part in deciding whether the BSP branch is read at all, so a local Module without such dependencies does not change how the remote requests resolve. A manifest depends on another local Module as local/<Name>, which modules.yaml must declare; a Module from a source cannot depend on a local Module.
  • When the dependencies of a local Module changed in the working tree, a plain xrobot setup keeps the locked commits, resolves the closure again and updates the lock; --frozen and --offline fail.
  • Modules/CMakeLists.txt adds each local Module by its path, with quotes, backslashes and $ escaped and a folder on another drive written as an absolute path.
  • setup prints Resolved N Module commits and M local Modules.

Also changes remote Modules

  • When the closure is resolved again (a request added or removed, --update of another Module, a local Module's dependencies changed), a locked explicit ref that still satisfies its request keeps its locked commit. Before, a branch that had moved upstream made setup fail and ask for xrobot setup --update <Module>.

Compatibility

  • The version is 1.0.2. xrobot 1.0.1 rejects a path: item in modules.yaml with an error, so a BSP that uses local Modules pins xrobot: 1.0.2. Projects without local Modules get byte-identical xrobot.lock, Modules/CMakeLists.txt and User/xrobot_main.hpp (compared step by step against dev).

Docs

Open points

  • module add --path does not change the xrobot: pin of the BSP.
  • With local Modules in the project, a lock entry that nothing requests is removed by a plain setup rather than reported: the lock does not record which Module brought an entry in.

All tests pass locally on Windows with Python 3.12 (585 run, 48 skipped); ruff check and ruff format --check pass; CI passes.

A BSP can use a Module straight from a folder, without a Git repository or a
Source of its own: `- path: <folder>` in Modules/modules.yaml, relative to the
BSP root or absolute. The Module id is local/<folder name>; the owner local is
reserved, and a Source that lists it is rejected. xrobot.lock records these
Modules under an optional local key with their paths; the modules key keeps
its meaning, and a lock without local Modules is unchanged byte for byte.

A folder inside the BSP is committed with the BSP repository and pinned by its
commits, so a header that the BSP's .gitignore ignores fails. A folder outside
the BSP, or one inside it with its own .git, only warns, also with --frozen.
The dependencies of a local Module follow the BSP branch like the requests of
modules.yaml; a Module from the Sources cannot depend on a local Module, and a
local dependency must be declared in modules.yaml. When a local Module gains
or drops a dependency, a plain setup keeps the locked commits and resolves the
closure again, while --frozen and --offline fail.

`xrobot module add --path` adds such a request, `--leave-local` refuses a local
Module, and describe reports kind, path and in_tree for each locked Module.
The Module id comes from the last segment of the written path, while the
primary header is read from the resolved folder. A wrong case on a
case-insensitive file system, or a symlink to a folder with another name,
gave an id that did not match the header: setup succeeded on that machine
and wrote a lock that fails where the file system is case-sensitive.

local_folder now compares the two names and fails with both in the
message. module add --path runs the same check when the folder exists.
…hanged

With a local Module in the project, a plain setup treated every lock
validation error as drift and resolved again, so a lock that lacked a
remote root or held a wrong ref was rewritten instead of reported.

validate_locked_graph now raises LocalDrift only for a dependency that a
local Module's manifest requests and the lock lacks or holds at another
ref, and for a leftover entry when local Modules exist (the lock does not
record which Module brought an entry in). Every other inconsistency
fails as it does without local Modules. --frozen and --offline fail in
all cases, as before.
The primary header of a Module folder was spelled out in eight places,
the last segment of a local Module path was computed in two, and the two
walks over local Modules each read the working-tree header and built the
same "not declared" error.

module_header and last_segment now own the two rules, and local_depends
and undeclared_local serve both walks. Behaviour and messages are
unchanged.
…describe

Adds tests for a cycle between local Modules while resolving and after
locking, --release-ref with a local Module, six malformed forms of the
local section of the lock, the stale generated file after a header edit,
and the describe fields of local and git Modules. The --update test now
proves that the dependencies of a local Module are resolved again.
Every local Module cost two git subprocesses in setup: one to learn
whether the BSP is a repository and one check-ignore. On Windows that is
about 70 ms each, 1.4 s for twenty local Modules.

The repository test is dropped: outside a repository check-ignore fails
and already reads as not ignored. The resolver asks git once for the
primary headers of all declared local Modules that are inside the BSP
and have no .git of their own, and keeps the answers. Errors, warnings
and their order are unchanged.

Setup with 20 local Modules: 50 git calls and 1637 ms before, 11 calls
and 252 ms now (Windows 11, median of 15 runs).
- `module add --path` writes the path on one line, checks the folder and
  its primary header, and takes a relative path from the current directory
  first, like -c; it exits with 2 again when neither a request nor --path
  is given.
- The local key of xrobot.lock copies the path requests: checking out the
  lock uses the folders of modules.yaml, --frozen fails when the copy
  differs, and a plain setup rewrites it. locked_modules reports a
  malformed entry as broken instead of raising.
- The root context is decided as before; the dependencies of local Modules
  count as requests of modules.yaml. A local Module without dependencies
  no longer changes how remote requests resolve, and --update outside Git
  fails for a local dependency that follows the BSP branch.
- When the closure is resolved again, a locked explicit ref that still
  satisfies its request keeps its commit instead of failing because the
  branch moved upstream. This applies to remote Modules as well.
- A path cannot be the BSP root, Modules/ or a folder around the BSP, and
  cannot hold a semicolon or control character; CMake receives quotes,
  backslashes and $ escaped, and a folder on another drive as an absolute
  path.
- The .gitignore check covers every source, header and CMake file of the
  local Modules in one `git check-ignore --stdin -z` call and names the
  line that lets the outermost ignored folder through.
- A manifest cannot write {path}; modules.yaml takes {path} only.
- Resolver, write_cmake and locked_modules use the same BSP root when
  Modules/ is a link.
- setup counts the local Modules it resolved.
@Jiu-xiao
Jiu-xiao merged commit ebb2fa0 into dev Nov 10, 2026
37 checks passed
@Jiu-xiao
Jiu-xiao deleted the feat/local-modules-20261109 branch November 11, 2026 00:07
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant