Skip to content

feat(agentd): an image-set default user for execs that omit user (#595) - #600

Merged
laithalsaadoon merged 5 commits into
mainfrom
feature/agentd-default-exec-user
Oct 12, 2026
Merged

feat(agentd): an image-set default user for execs that omit user (#595)#600
laithalsaadoon merged 5 commits into
mainfrom
feature/agentd-default-exec-user

Conversation

@laithalsaadoon

@laithalsaadoon laithalsaadoon commented Oct 12, 2026 •
edited
Loading

Copy link
Copy Markdown
Owner

What and why

An image can now set AGENTD_DEFAULT_USER to the user that an exec or run --exec start runs as when the request has no user (#595). Before this, such a start always ran as the daemon's own user, which is root. A caller that left out user handed the workload guest root.

  • The daemon reads the variable once at startup and keeps it as written: a user name, or a uid as digits. Each start that omits user resolves it to a row of the guest's /etc/passwd, by name or by the uid its digits spell. The child gets that row's uid, primary gid, and HOME/USER/LOGNAME, so AGENTD_DEFAULT_USER=1000 and AGENTD_DEFAULT_USER=agent run the same way. Resolution happens per request, so a user that an earlier exec created resolves on the next start (AGENTD-22).
  • A request's own user runs as given, root and 0 included (AGENTD-23). An authorized caller can still ask for root. A request's own digits keep protocol 1's meaning: a bare uid in the daemon's group, with or without a row (AGENTD-16).
  • If no row matches (an unknown name, digits with no row, or an empty value), each start without a user gets 400 unknown_user (AGENTD-24). The detail names AGENTD_DEFAULT_USER, and for digits it says the image needs a passwd row. It doesn't fall back to root or to a uid in root's group. The daemon still boots, and a start that sets its own user still runs.
  • Uploads don't follow the variable. What PUT /v1/fs/tar and PUT /v1/fs/file create stays root's, as on main, so the default user can read the tree run . --exec and cp upload but can't write into it. An exec that must write there runs with user: 0, or the image or a root exec hands the tree over with chown first. PROTOCOL.md's "Upload ownership" section, TRUST.md, the guest Dockerfile guide and the changelog say so.
  • When the variable is unset, nothing changes. A start without user doesn't read passwd.
  • Workload hook handlers ignore the setting and keep running as root. PROTOCOL.md and TRUST.md now say so.
  • The agent recipe's chown of /workspace and a sync's in-guest rm -rf need root, and both now send user: 0. On an image that sets the variable, they'd otherwise run as its default user, where they can fail. So an image that sets the variable needs a client at this release or later, and the changelog says so.

The default isn't reported on /v1/health. In microvms-protocol 1.0.0, callers construct Health field by field. A new field fails semver:check (measured: constructible_struct_adds_field). Instead, the daemon logs the value at startup to the image's log group. The only wire schema change is the StartRequest.user description.

The live conformance suite doesn't cover AGENTD-22..24, which waive the live layer: the suite's images don't set a default, and an image that did would change who every other live exec runs as. A local container run of the release binary covers the numeric default instead (see Evidence).

Size: 546 changed lines of product code, 363 of them unit tests in the files this edits. The setting, its per-request resolution to a passwd row, the refusal, and the user: 0 pins belong in one change. Without the row's gid, a numeric default runs workloads in root's group. Without the pins, any image that sets the variable runs the agent recipe's chown and a sync's deletions as a non-root user.

Evidence

  • mise run check passes on 7558ada in 230 s: 1,813 Rust tests pass across 69 test binaries, 0 fail, and 28 carry #[ignore]. Prose ends at 0 errors and 4,077 warnings, and Vale reports no new warning on any edited doc against the merge base.

  • cargo fmt --all -- --check and cargo clippy --all-targets -- -D warnings, inside mise run check.

  • cargo run -p agentd --bin schema -- --check: regenerated. Only StartRequest.user's description changed in docs/schema.json, and two --user help strings changed in docs/manifest.json.

  • cargo build --release -p agentd --target aarch64-unknown-linux-musl

  • Live container, public.ecr.aws/amazonlinux/amazonlinux:2023-minimal with the x86_64 musl release build of 7558ada's daemon as root, agent:x:1000:1000::/workspace:/bin/bash appended to /etc/passwd, and AGENTD_DEFAULT_USER=1000. Every exec below omits user unless shown:

    $ exec id
uid=1000(agent) gid=1000(agent) groups=1000(agent)
$ exec ls -ld /root; ls /root
ls: cannot open directory '/root': Permission denied
$ exec [user: 0] id
uid=0(root) gid=0(root) groups=0(root)

    Restarted with AGENTD_DEFAULT_USER=4242 and no row for 4242:

    $ exec id
HTTP 400 {"error":"unknown_user","detail":"the request names no user, and the default user \"4242\" that AGENTD_DEFAULT_USER sets is a uid with no row in the guest's /etc/passwd (the image needs a passwd row for it)"}

  • Mutation testing: tools/check-mutants.py --diff (the script ci:mutants runs) over this branch's changed lines of exec_start.rs tested 6 mutants. It caught 2, 4 were unviable, and it missed none. Earlier, the first CI run failed shard 3 of 4 in cargo-mutants' unmutated baseline, because guards::parity::the_cli_answers_the_shared_case_corpus_against_its_fakes spawns microvms-core's parity-tunnel-peer example, which a mutant's cargo test -p microvms-cli never builds. a23b15b makes ci:mutants depend on parity:fixtures, and mise run ci:mutants --base origin/main -- --shard 3/4 --in-place then passed its baseline locally.

  • mise run spec and mise run spec:core. spec exits 0 with no error findings. Warnings go from 102 to 112 against the merge base, all in the lint tier, all on the three new sentences. spec:core exits 1 on one GTWR_R26_ABSOLUTE error ("never") in TRAP-4. That's in verify/spec/core.symspec.json, which this branch leaves alone, and its inputs are byte-identical to main's.

  • Live conformance run: not run. See What and why.

Parity

  • No public name changed on any surface (core, CLI commands, microvms.pyi, index.d.ts).
  • verify/parity/capabilities.toml doesn't change, because no surface gains or loses a capability. The setting belongs to the daemon, which reads it from the image. The user option keeps its type everywhere it appears: with_user in core, user= in Python, and user? in Node. mise run parity:check passes.

Guards

The merge base can't compile the new agentd and model tests, because they use Config::default_user, Guest::default_user, and the model's new types. Those tests get hand-written faults in verify/guards/faults/default-user.toml instead. The microvms-app tests install_access_uploads_then_chowns_as_root and a_hostile_deletion_path_is_refused_not_executed do compile on the base, and mise run fail-to-pass -- --emit default-user wrote their entries. Their patches are regenerated against f410ec2.

mise run guards:fire -- --only <id> over every default-user entry, on 7558ada with a clean tree, printed the lines below, then guards: restored, every command passes again:

fired: default-user-ignored (6.2 s)
fired: default-user-ignored-fuzz (5.9 s)
fired: default-user-dropped-at-build (4.4 s)
fired: default-user-passwd-unread (4.8 s)
fired: default-user-overrides-request (3.0 s)
fired: default-user-overrides-request-handler (2.4 s)
fired: default-user-unknown-runs-as-daemon (4.7 s)
fired: default-user-numeric-keeps-daemon-gid (3.2 s)
fired: default-user-numeric-keeps-daemon-gid-fuzz (5.6 s)
fired: default-user-digits-without-row-run (2.8 s)
fired: default-user-digits-without-row-run-handler (2.6 s)
fired: model-exec-start-numeric-default-as-request (4.4 s)
fired: default-user-empty-read-as-unset (3.5 s)
fired: default-user-env-unread (2.7 s)
fired: default-user-hook-handler-demoted (6.0 s)
fired: model-exec-start-default-ignored (8.2 s)
fired: f2p-install-access-uploads-then-chowns-as-root (9.9 s)
fired: f2p-a-hostile-deletion-path-is-refused-not-executed (10.0 s)

Follow-ups

  • Upload ownership under a default user is Uploads stay root-owned, so AGENTD_DEFAULT_USER can't write a synced tree #604: give the default user what an upload creates, through a walk that refuses a symlink already in the path and ties each chown and chmod to the inode the upload created.
  • Report the default on /v1/health once the protocol can take a field, either through a semver:check allowance in a minor release or a #[non_exhaustive] Health in 2.0.
  • Add a live check for AGENTD-22..24 on a dedicated image that sets AGENTD_DEFAULT_USER, so the waived live layer has a check.
  • agentd: an image-set default user for execs and run-path starts #595's second step, an opt-in drop of CAP_SYS_ADMIN and similar capabilities from the bounding set, is agentd: opt-in capability bounding-set drop after identity repair #599, with the design options weighed and the live measurements to take first.
  • wrap_dockerfile could set AGENTD_DEFAULT_USER from a task's last USER. A wrapped image would then run workloads as its author's user.
  • A tar whose hard-link member points at its own name answers 500 ENOENT, because create_hard_link unlinks the destination before it links the source, which is the same entry. GNU tar writes one when a file is listed twice (tar -cf x.tar dir dir/file). This predates the change.

Platform claims

  • Not applicable: this changes no claim about AWS behavior.

Scope

  • This is not an orchestrator or a fork implementation
    (see docs/STRATEGY.md).

laithalsaadoon added a commit that referenced this pull request Oct 12, 2026
@laithalsaadoon
ci: build the parity tunnel fixture before the mutants job's baseline
ccdb6a4 
The CLI's `guards::parity::the_cli_answers_the_shared_case_corpus_against_its_fakes` spawns
microvms-core's `parity-tunnel-peer` example from the test binary's target directory. A mutant's
`cargo test -p microvms-cli` never builds another package's example, so a change with mutants in
microvms-cli failed cargo-mutants' unmutated baseline (exit 4, "build the fixture with mise run
parity:fixtures"), as shard 3 of 4 did on #600. `ci:mutants` now depends on `parity:fixtures`,
which builds the example into the target the `--in-place` run tests from. Shard 3 of 4 run
locally with this: baseline ok in 31 s build + 9 s test, 1 mutant, unviable.
@laithalsaadoon laithalsaadoon mentioned this pull request Oct 12, 2026
Merged
3 tasks done
laithalsaadoon added a commit that referenced this pull request Oct 12, 2026
@laithalsaadoon
ci: build the parity tunnel fixture before the mutants job's baseline
52568b2 
The CLI's `guards::parity::the_cli_answers_the_shared_case_corpus_against_its_fakes` spawns
microvms-core's `parity-tunnel-peer` example from the test binary's target directory. A mutant's
`cargo test -p microvms-cli` never builds another package's example, so a change with mutants in
microvms-cli failed cargo-mutants' unmutated baseline (exit 4, "build the fixture with mise run
parity:fixtures"), as shard 3 of 4 did on #600. `ci:mutants` now depends on `parity:fixtures`,
which builds the example into the target the `--in-place` run tests from. Shard 3 of 4 run
locally with this: baseline ok in 31 s build + 9 s test, 1 mutant, unviable.
@laithalsaadoon
feat(agentd): an image-set default user for execs that omit user (#595
28f31e9 
)

A start request that omits `user`, from `exec` or `run --exec`, runs as the
daemon's uid, which is root, so every workload is guest root unless each caller
remembers to demote it. The daemon now reads `AGENTD_DEFAULT_USER` from the
image's environment at startup. A start that omits `user` resolves that string
at each request exactly as it would resolve it in `user` (AGENTD-22), a
request's own `user` still runs as given, `root` and `0` included (AGENTD-23),
and a value the guest's /etc/passwd lacks refuses the start as `unknown_user`
with a detail that mentions the variable (AGENTD-24). Unset, nothing changes.
Workload hook handlers still run as root.

`/v1/health` doesn't report the default: `Health` is constructible field by
field, and a new field fails `semver:check` (`constructible_struct_adds_field`,
measured), so the daemon logs it at startup instead.

The agent recipe's `chown` of /workspace and a sync's in-guest deletions need
root, so they ask for uid 0 by number instead of omitting `user`.

The model, the Gherkin tier, the fuzz harness and the handler tests cover the
three new requirements; the live layer is waived because the live images set
no default.
@laithalsaadoon
ci: build the parity tunnel fixture before the mutants job's baseline
a23b15b 
The CLI's `guards::parity::the_cli_answers_the_shared_case_corpus_against_its_fakes` spawns
microvms-core's `parity-tunnel-peer` example from the test binary's target directory. A mutant's
`cargo test -p microvms-cli` never builds another package's example, so a change with mutants in
microvms-cli failed cargo-mutants' unmutated baseline (exit 4, "build the fixture with mise run
parity:fixtures"), as shard 3 of 4 did on #600. `ci:mutants` now depends on `parity:fixtures`,
which builds the example into the target the `--in-place` run tests from. Shard 3 of 4 run
locally with this: baseline ok in 31 s build + 9 s test, 1 mutant, unviable.
@laithalsaadoon
fix(agentd): resolve the default user to a passwd row and give it upl…
d6a33c9 
…oads (#595)

A numeric AGENTD_DEFAULT_USER was resolved the way a request's own digits
are: a bare uid in the daemon's group. AGENTD_DEFAULT_USER=1000 ran workloads
as uid 1000 in gid 0, so they could read and write whatever root's group can.
The default now resolves to a row of the guest's /etc/passwd, by name or by
the uid its digits spell. The child gets that row's uid, primary gid, HOME,
USER and LOGNAME. A default with no row, digits included, refuses each start
without a `user` as 400 unknown_user. The detail names the variable and says
the image needs a passwd row (AGENTD-24). A request's own digits still run as
a bare uid in the daemon's group (AGENTD-16). The model gains numbered and
unnumbered defaults, and the fuzz harness, the BDD tier and the handler tests
cover both.

Uploads stayed root's, so a nameless exec under the default user couldn't
write the tree `run . --exec` and `cp` had just uploaded. With the variable
set, what PUT /v1/fs/tar and PUT /v1/fs/file create (files, directories,
symlinks, and created parents) now belongs to the default user's uid and
primary gid. What already existed keeps its owner, and a replaced file or
symlink keeps the owner of the entry it replaced. Each chown is
fchownat(fd, "", AT_EMPTY_PATH) on the entry's own descriptor, never a path:
tar members through the extraction's openat2 walk, and a symlink opened as
itself and checked to be a symlink first. A default with no row refuses the
upload with the same 400 unknown_user before the body is read. Unset, uploads
stay root's. The fs API has no directory-creation route, so these two routes
are the whole surface. The schema lists the new 400 on both.

The changelog says an image that sets the variable needs a client at this
release or later, since an older one sends the recipe's chown and a sync's
deletions with no `user`.

fs.rs's directory and link open flags become `.union()` consts that every
open shares, so cargo-mutants has no equivalent `|` mutant to report. The harden-agentd and default-user
FAIL_TO_PASS patches are rebased onto the new fs.rs, and new seeded faults in
verify/guards/faults/default-user.toml cover each rule above.
@laithalsaadoon
docs: the DS-0002 reason and the MMDS block name the default user
7388f52 
#594 merged statements that an exec naming no user runs as root. With AGENTD_DEFAULT_USER set it runs as the image's user, so the trivy exception and TRUST.md's MMDS paragraph say when an exec runs as root: with user 0, or without user on an image that doesn't set the variable.
@laithalsaadoon
laithalsaadoon force-pushed the feature/agentd-default-exec-user branch from ccdb6a4 to 7388f52 Compare October 12, 2026 22:13
@laithalsaadoon laithalsaadoon mentioned this pull request Oct 12, 2026
Closed
@laithalsaadoon
revert(agentd): keep uploads root-owned under a default user (#595)
7558ada 
d6a33c9 gave the default user what PUT /v1/fs/tar and PUT /v1/fs/file
create. Review found that this hands the default user files outside the
named tree. The owned walk follows a symlink that's already there with only
RESOLVE_NO_MAGICLINKS. A workload running as the default user can swap a
directory it owns from an earlier sync for a symlink to a directory root
reads, and the next upload to that path extracts into the link's target and
gives every new entry to the default user. The file route has the same hole
twice: a planted symlink, and a race between the walk's descriptor and the
reopen by path.

This takes that half back out. fs.rs, schema.rs and agentd's Cargo.toml go
back to main's versions, and so do the ten harden-agentd FAIL_TO_PASS
patches rebased onto the owned fs.rs. The upload faults leave
default-user.toml. Uploads don't read AGENTD_DEFAULT_USER again, so a
default with no row no longer refuses one. The agent recipe's and the
sync's comments go back to saying the daemon writes uploads as root.

PROTOCOL.md's Upload ownership section, TRUST.md, the guest Dockerfile
guide and the changelog now state the limit. Uploads stay root's: a tar's
modes are masked to 0755 at most, and PUT /v1/fs/file creates its parents
0755 and its file 0644 unless the request sets a mode. So the default user
can read an uploaded tree but can't write into it. An exec that must write
there runs with user 0, or a root chown hands the tree over first. The
guide no longer offers the variable as the default for every exec and
`run --exec`. Safe upload ownership is its own issue.

The numeric default's row and gid (Finding 1) and the changelog's
client-version sentence (Finding 3) stay as they were. default_user_row is
private again now that fs.rs doesn't call it. The two default-user
FAIL_TO_PASS patches are regenerated against f410ec2.
@laithalsaadoon
laithalsaadoon marked this pull request as ready for review October 12, 2026 23:10
Copilot AI balanced review requested due to automatic review settings October 12, 2026 23:10
@laithalsaadoon
laithalsaadoon merged commit e209162 into main Oct 12, 2026
35 checks passed
Copilot AI reviewed Oct 12, 2026

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

@laithalsaadoon laithalsaadoon mentioned this pull request Oct 13, 2026
Closed
35 of 45 tasks
bonk-ai Bot added a commit that referenced this pull request Oct 13, 2026
@bonk-ai
docs: drop the debt and debugging entries #576, #579 and #583 fixed
b73b04b 
tech-debt.md loses items 5 (napi generator pin, fixed by #576) and 10
(lifecycle commentary, fixed by #579) and the bullets that cited them;
item 8 now names #576's agreement test and the proof-runner copy it
leaves out. debugging-guide.md drops the Serve-not-Future entry that
#583's accept loop retired. 15 citations #600 and #601 moved on those
pages are re-pointed by exact text match. The see-also footers on 19
pages drop their hand-written citation counts (AGENTS.md, Counts in
docs), and .vale/warnings.json lowers 25 ceilings to this tree's counts.

docs:check EXIT=0 (240 s), prose:check EXIT=0 (0 errors, 4069 warnings).
@bonk-ai bonk-ai Bot mentioned this pull request Oct 13, 2026
Merged
2 tasks done
bonk-ai Bot added a commit that referenced this pull request Oct 13, 2026
@bonk-ai
docs: drop the debt and debugging entries #576, #579 and #583 fixed (#…
683cd37 
…607)

tech-debt.md loses items 5 (napi generator pin, fixed by #576) and 10
(lifecycle commentary, fixed by #579) and the bullets that cited them;
item 8 now names #576's agreement test and the proof-runner copy it
leaves out. debugging-guide.md drops the Serve-not-Future entry that
#583's accept loop retired. 15 citations #600 and #601 moved on those
pages are re-pointed by exact text match. The see-also footers on 19
pages drop their hand-written citation counts (AGENTS.md, Counts in
docs), and .vale/warnings.json lowers 25 ceilings to this tree's counts.

docs:check EXIT=0 (240 s), prose:check EXIT=0 (0 errors, 4069 warnings).

Co-authored-by: bonk-ai[bot] <269762587+bonk-ai[bot]@users.noreply.github.com>
bonk-ai Bot added a commit that referenced this pull request Oct 14, 2026
@bonk-ai
docs: re-point the source citations #600 and #601 moved
e19a0e2 
604 path:line citations on 15 hand-written pages pointed a few lines off after
#600 and #601 moved their source lines. Each move is the doc line's blame-commit
block found at HEAD by exact text and by git's own diff line mapping, or a
read-only audit against the sentence for the 20 rows neither settled.

state-machines.md's VmState section now says what ResumeRequested does after
SuspendedVmEnded (client.rs:494-499, 509-512): the service refuses the request
and the model writes Terminated.
@bonk-ai bonk-ai Bot mentioned this pull request Oct 14, 2026
Merged
2 tasks done
bonk-ai Bot added a commit that referenced this pull request Oct 14, 2026
@bonk-ai
docs: re-point the source citations #600 and #601 moved (#612)
2eb80b6 
604 path:line citations on 15 hand-written pages pointed a few lines off after
#600 and #601 moved their source lines. Each move is the doc line's blame-commit
block found at HEAD by exact text and by git's own diff line mapping, or a
read-only audit against the sentence for the 20 rows neither settled.

state-machines.md's VmState section now says what ResumeRequested does after
SuspendedVmEnded (client.rs:494-499, 509-512): the service refuses the request
and the model writes Terminated.

Co-authored-by: bonk-ai[bot] <269762587+bonk-ai[bot]@users.noreply.github.com>
@bonk-ai bonk-ai Bot mentioned this pull request Oct 14, 2026
Merged
3 tasks done
This was referenced Oct 14, 2026
Closed
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

Copilot code review Copilot
Copilot review effort
Applies to this pull request for everyone.Learn more about Copilot code review.
Copilot left review comments

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@laithalsaadoon