feat(bundle): a registry image with its build context beside it, and bundle init --image-ref

AGENTS.md

@@ -122,7 +122,7 @@ and deserialized through a registry.
 | `openenv/` | `build_app(bundle)`: the [OpenEnv](https://github.com/openenv/OpenEnv) server for a bundle's tasks, each session playing one episode through the Episode API, with the envs' MCP tools, an added `end_turn` tool and the task's verifier score as the reward. It prepares bundles as `agent-env run` does (`prepare_bundle(played=True)`, without the agent the client plays) and keeps namespace routing on for the app's life. `openenv` comes with the `openenv` extra, a floor with no cap; this package and `training/trl.py` are the only ones that import it. Each task's description in the Task API names its `toolset` (the played agent's envs, role, and whether it gets `bash`). |
 | `training/` | One module per outside trainer, each behind its own extra, and `_tools.py`, which turns an env's tools into typed, documented Python methods from their JSON Schema, as transformers' `get_json_schema` reads them. `trl.py` (`trl` extra): `AgentEnv(url)` over a server `openenv/` makes, with `dataset` (a row per task) and `factory` (TRL's `environment_factory`: a generated class per toolset whose methods are the tools, a background `reset` and an owned `get_reward`); `AgentEnv.local(bundle)` serves the bundle in process. |
 | `runner/` | The `[runner]` seam: `Runner.submit()` returns a `RunHandle` (`run_id`, `instance_id`) before the run starts; `LocalRunner` is built in. `runner/store.py` is the run index every runner writes and the explorer reads. |
-| `bundle/` | A bundle is a folder of entities (`envs/`, `agents/`, `artifacts/`, `skills/`, `tasks/<name>.json`, `evals/<name>.toml`) that runs without a registry: `agent-env run` plans it, writes it into the `@local` namespace (reusing each version whose inputs haven't changed) and runs its evals or tasks; `agent-env bundle init` writes a stored task or eval as one, with every env, agent and artifact it uses in its own folder (each image as its build context, pruned to what its Dockerfile reads and pinned to the Python and npm versions of the stored image) so it runs with no store; `agent-env bundle register` writes a folder into the configured stores under registry ids (`<prefix><name>`), each task and eval naming the versions it reads and each image it builds pushed, through a ledger in the target store so registering again writes only what changed. |
+| `bundle/` | A bundle is a folder of entities (`envs/`, `agents/`, `artifacts/`, `skills/`, `tasks/<name>.json`, `evals/<name>.toml`) that runs without a registry: `agent-env run` plans it, writes it into the `@local` namespace (reusing each version whose inputs haven't changed) and runs its evals or tasks; `agent-env bundle init` writes a stored task or eval as one, with every env, agent and artifact it uses in its own folder (each image as its build context, pruned to what its Dockerfile reads and pinned to the Python and npm versions of the stored image, or, given `--image-ref ID=REF`, as that registry image pinned by digest with its context beside it, `image = { ref, dockerfile }`, which a run pulls and builds only when it can't) so it runs with no store; `agent-env bundle register` writes a folder into the configured stores under registry ids (`<prefix><name>`), each task and eval naming the versions it reads and each image it builds pushed, through a ledger in the target store so registering again writes only what changed. |
 | `eval/` | `Eval` (`eval`), a versioned list of task references, and its store. |
 | `plugins/` | Entry-point plugins (see Extension points): the group names, `inventory()`, which `agent-env plugin list/show/check` report, `load_failures` and `settings`. The top level is public; the underscored modules are internal. |
 | `explorer/` | Optional local web UI: `agent-env up`, binds loopback `:8234`. |

src/agent_env/artifact/artifacts/docker_image.py

@@ -217,8 +217,9 @@ def put_ref(cls, id: str, *, description: str, image_name: str, earlier_pin: str
                 dockerfile_path: str | None = None) -> "DockerImageArtifact":
         """Register ``image_name``, an image already in a registry, which a sandbox pulls; nothing is built, pushed or
         saved. A tag is pinned to the digest the registry serves now, as ``name:tag@sha256:...`` (``earlier_pin`` kept
-        when it pins the same ref). Given the image's build context and ``image_digest``, the tag stays the name, the
-        digest is recorded beside it, and the context is uploaded for a sandbox that can't pull the image to build it.
+        when it pins the same ref). Given the image's build context, the name is recorded without its digest, the
+        digest beside it (``image_digest``, or the one the registry pins it to), and the context is uploaded for a
+        sandbox that can't pull the image to build it.
 
         ``image_name`` must name its registry, read with the configured image store's credentials where it holds them;
         one on this machine is refused unless ``id``'s image store is the local registry."""
@@ -235,25 +236,26 @@ def put_ref(cls, id: str, *, description: str, image_name: str, earlier_pin: str
                 and not isinstance(config.get_image_store_for(id), LocalRegistryImageStore)):
             raise ValueError(f"{image_name} is in a registry on this machine, which only an image in the local "
                              "registry's store, such as an @local one, can name")
-        if context_path is not None:
-            context = _context_of(Path(context_path), dockerfile_path)
-            store = get_artifact_store()
-            prefix = store.attempt_prefix("docker_image", id)
-            return store.put_document(cls(
-                id=id,
-                description=description,
-                image_name=image_name,
-                image_digest=image_digest,
-                build_context_object_url=_upload_context(config.get_object_store_to_write(prefix, id), prefix, context),
-                dockerfile_path=context.dockerfile,
-                source_digest=context.source_digest(None),
-            ))
-        if earlier_pin is not None and pinned_from(earlier_pin, image_name):
-            pinned = earlier_pin
-        else:
-            pinned = pin_digest(image_name, config.get_image_store().auth(image_name))
+        if context_path is None or image_digest is None:
+            if earlier_pin is not None and pinned_from(earlier_pin, image_name):
+                pinned = earlier_pin
+            else:
+                pinned = pin_digest(image_name, config.get_image_store().auth(image_name))
+            if context_path is None:
+                return get_artifact_store().put_document(cls(id=id, description=description, image_name=pinned))
+            image_name, _, image_digest = pinned.rpartition("@")
+        context = _context_of(Path(context_path), dockerfile_path)
         store = get_artifact_store()
-        return store.put_document(cls(id=id, description=description, image_name=pinned))
+        prefix = store.attempt_prefix("docker_image", id)
+        return store.put_document(cls(
+            id=id,
+            description=description,
+            image_name=image_name,
+            image_digest=image_digest,
+            build_context_object_url=_upload_context(config.get_object_store_to_write(prefix, id), prefix, context),
+            dockerfile_path=context.dockerfile,
+            source_digest=context.source_digest(None),
+        ))
 
     @classmethod
     def put_context(cls, id: str, *, description: str, context_path: str, dockerfile_path: str,

src/agent_env/bundle/_images.py

@@ -34,15 +34,20 @@ class ImageJob:
     problem: str | None = None  # why it can't be written
     locked: Locked | None = None
     unlocked: str | None = None  # why its versions weren't pinned
+    ref: str | None = None  # the registry image the folder names, given for it, its context beside it
 
 
 def context_groups(jobs: list[ImageJob]) -> list[list[ImageJob]]:
     """``jobs`` grouped by the stored context they're built from: the same object when the store tells its content
-    (an ETag), else the same URL."""
+    (an ETag), else the same URL. A context in a store this config can't read is grouped by its URL, and its download
+    says why it can't be written."""
     groups: dict[tuple, list[ImageJob]] = {}
     for job in jobs:
         url = job.artifact.build_context_object_url
-        metadata = get_config().get_object_store_at(url).get_object_metadata_at(url)
+        try:
+            metadata = get_config().get_object_store_at(url).get_object_metadata_at(url)
+        except ValueError:
+            metadata = None
         job.size = metadata.size if metadata is not None else None
         key = ("etag", metadata.etag, metadata.size) if metadata is not None and metadata.etag else ("url", url)
         groups.setdefault(key, []).append(job)
@@ -91,6 +96,13 @@ def write_images(groups: list[list[ImageJob]], staging: Path, scratch: Path,
     for job in placed:
         sharing.setdefault((job.folder, job.pruned.dockerfile), []).append(job)
     for (folder, dockerfile), jobs in sharing.items():
+        if pulled := [job.key for job in jobs if job.ref is not None]:
+            # A run pulls an image given its ref, and a deploy that can't rebuilds it from this Dockerfile as written.
+            for job in jobs:
+                if job.ref is None:
+                    job.unlocked = (f"its Dockerfile is also the build context of {', '.join(pulled)}, which a run "
+                                    "pulls, so a build resolves them afresh")
+            continue
         _lock(jobs, staging / folder, dockerfile)
 
 

src/agent_env/bundle/_import_report.py

@@ -75,6 +75,7 @@ class WrittenImage:
     pruned: Pruned  # what its folder holds of the context
     locked: Locked | None  # what was pinned to the stored image's versions
     unlocked: str | None  # why nothing was
+    ref: str | None = None  # the registry image a run pulls, given for it, the context kept beside it
 
 
 @dataclass(frozen=True)
@@ -156,6 +157,7 @@ def provenance(source: dict, imported: BundleImport) -> str:
         "written": [{"folder": entity.folder, "kind": entity.kind.value, "type": entity.type, "id": entity.id,
                      "version": entity.version} for entity in imported.written],
         "images": [{"folder": image.folder, "key": image.key, "id": image.artifact, "version": image.version,
+                    **({"ref": image.ref} if image.ref else {}),
                     "dockerfile": image.pruned.dockerfile, "dockerfile_found": image.found}
                    for image in imported.images],
         "store": [{"where": pin.where, "kind": pin.kind.value, "id": pin.id, "version": pin.version, "why": pin.why}
@@ -170,8 +172,10 @@ def readme(description: str, imported: BundleImport) -> str:
     held = ("Every env, agent and artifact it uses is written here, so `agent-env run` runs it with no store"
             + (", except what *Still needs the store* lists, which it reads from the store this was imported from."
                if imported.pinned else "."))
+    built = [_BUILT_IMAGES] if any(image.ref is None for image in imported.images) else []
+    pulled = [_PULLED_IMAGES] if imported.pulled or any(image.ref for image in imported.images) else []
     lines = [description, "", f"Imported with agent-env {package_version('agentenv-framework')} on {stamp}. {held}", "",
-             *_RUN_IT, *_needs_lines(imported), "", *_SHARE_IT, ""]
+             *_RUN_IT, *_needs_lines(imported), "", *built, *pulled, "", *_SHARE_IT, ""]
     contents = _sections([("Tasks", [f"{task.path}: task {task.id} v{task.version}" for task in imported.tasks]),
                           ("Eval", [f"{imported.eval.path}: eval {imported.eval.id} v{imported.eval.version}"]
                            if imported.eval else [])])
@@ -186,8 +190,11 @@ def _image_line(image: WrittenImage) -> str:
     found = " (found: the image records no Dockerfile)" if image.found else ""
     largest = ", ".join(f"{name} ({size_text(size)})" for name, size in pruned.largest)
     whole = f"; all of its build context, since {pruned.whole}" if pruned.whole else ""
-    return (f"{image.folder}: {image.key} built from {pruned.dockerfile}{found}, {pruned.files} file(s), "
-            f"{size_text(pruned.size)}{whole}; largest: {largest}; {_locked_text(image)}")
+    held = f"{pruned.files} file(s), {size_text(pruned.size)}{whole}; largest: {largest}"
+    if image.ref:
+        return (f"{image.folder}: {image.key} pulled as {image.ref}, its build context beside it from "
+                f"{pruned.dockerfile}{found}, {held}")
+    return f"{image.folder}: {image.key} built from {pruned.dockerfile}{found}, {held}; {_locked_text(image)}"
 
 
 def _locked_text(image: WrittenImage) -> str:
@@ -223,11 +230,14 @@ def _listed(items: list[str]) -> str:
     f"- A model endpoint for its agents and judges: `{ENV_BASE_URL}` and `{ENV_API_KEY}`, or `[model] base_url` and "
     "`api_key` in `.agentenv/config.toml`.",
     "- An agent where a step names none: the configured default (`[agents] default_a2a_agent_id`).",
-    "",
+]
+_BUILT_IMAGES = (
     "Its images are built from the build contexts in their folders on the machine that runs it, pinned to the "
     "Python and npm versions of the images the store held, as *Images* says. To build one unpinned, take out the "
-    f"COPY and two ARG lines naming {posixpath.basename(IN_IMAGE)} after each FROM in its Dockerfile.",
-]
+    f"COPY and two ARG lines naming {posixpath.basename(IN_IMAGE)} after each FROM in its Dockerfile.")
+_PULLED_IMAGES = (
+    "An image *Images* says is pulled comes from its registry and is never built here. A build context beside one is "
+    "there to read or rebuild, and a deploy builds from it only when it can't pull the image.")
 _SHARE_IT = [
     "## Share it",
     "",
@@ -247,7 +257,8 @@ def _needs_lines(imported: BundleImport) -> list[str]:
     lines = [f"- The plugins it uses: `{'`, `'.join(plugins)}`."] if plugins else []
     if configured:
         lines.append("- The configuration that registers what *Needs beyond core* says configuration registers.")
-    registries = sorted({host for pulled in imported.pulled if (host := registry_host_from_ref(pulled.image))})
+    refs = [pulled.image for pulled in imported.pulled] + [image.ref for image in imported.images if image.ref]
+    registries = sorted({host for ref in refs if (host := registry_host_from_ref(ref))})
     if registries:
         lines.append(f"- Access to the registries its pulled images are in: `{'`, `'.join(registries)}`. agent-env reads "
                      "a private one only through its configured image store, such as an ECR store for an ECR registry.")