Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ All notable changes to OriginWeave are documented in this file. The format follo
### Added

- Failed controlled Agent Task runs now report whether their original browser process ended after shutdown, alongside temporary-profile cleanup; a failed task never becomes a pass merely because cleanup succeeded. If process observation itself fails, termination remains unproven. This covers the original browser process only, not all descendants or arbitrary browser recovery.
- Controlled pinned-Chromium Agent Task success now binds the ChromeDriver browser PID to its Linux `/proc/<pid>/stat` start-time identity and fails closed unless that exact root process terminates after session/driver shutdown; PID reuse counts only as termination of the original identity, and this does not yet prove termination of every Chromium descendant or process ownership outside the controlled runner.
- Controlled pinned-Chromium Agent Task success now binds the ChromeDriver browser root to its exact Linux `/proc/<pid>/stat` start-time identity, binds every still-live PID from the already sampled bounded Chromium root-plus-descendant set before shutdown, explicitly records descendants that already exited between the `/proc` lineage snapshot and identity capture, and fails closed unless every retained exact identity terminates after session/driver shutdown; root disappearance or identity change remains an error, PID reuse counts only as termination of the original identity, and this does not attest cgroup/task ownership, processes appearing only after the sample, or OS-wide orphan absence.
- Failed ordinary and forced-close Agent Task browser trials now retain credential-free temporary-profile cleanup evidence after bounded browser errors, and separate aggregate compatibility gates require cleanup proof from every trial rather than filtering unsuccessful trials out; this does not attest adversarial filesystem erasure, process termination, or arbitrary browser recovery.
- Failed Manifest V3 restart trials now retain credential-free temporary-profile cleanup evidence after bounded browser errors, including reviewed ChromeDriver process-teardown `TimeoutExpired` failures; successful trials record the same cleanup fact, and an aggregate compatibility gate requires teardown proof from every MV3 trial before repeatability acceptance without retaining exception messages or command paths; this does not attest adversarial filesystem erasure, browser-process termination, or cleanup outside the controlled temporary profile.
- Rust workspace for independently reusable core, policy, destination, network, TLS, resource, and evidence modules.
Expand Down
6 changes: 6 additions & 0 deletions docs/doctoring.md
Original file line number Diff line number Diff line change
Expand Up @@ -98,6 +98,12 @@ Process-observation errors leave termination unproven: a read failure escapes th

The release-record regression first failed because this changed failure path had no corresponding Unreleased entry. A controlled behavioral regression now exercises the real browser-pass and trial functions with no browser launch: observed exit, surviving identity and observation error all remain failed trials, driver shutdown is required, and private exception text is absent from returned evidence. These controlled tests are not Linux process or pinned-Chromium runtime evidence; exact-head hosted compatibility remains required.

### Process-set stack integration evidence

The #144 ordinary parent integration preserves the production runner and process-set tests from `09f2e087d0c20fe81386c18099e739a9e611a8ad` while adopting #143 `44fd9a450f864feff5cf2ba2883425a71ba10b9b`. The old child had only three failure-path contracts instead of the parent's five; the missing release-record and real browser-pass/trial regressions now run with the child process-set contracts. The CHANGELOG-only conflict keeps the parent's failed-task root-cleanup record and the child's expanded success-path process-set record; the latter already includes the earlier root-only success contract. No runtime behavior or deadline is changed.

The prior informational review limits remain explicit: descendants are bound to start times read after lineage sampling, so a reused PID in that gap may conservatively fail a trial; root and process-set waits have separate bounded budgets, with one shared deadline only inside the set waiter; and the pre-shutdown exit-count bound is a defensive evidence invariant. None of these controlled local contracts attests cgroup ownership, later processes, OS-wide orphan absence or exact-head pinned-Chromium acceptance. The inherited failure-path regression keeps observer errors unproven and task failures unsuccessful even when temporary-profile cleanup succeeds.

## References

Amazon Web Services. (n.d.). *Set up the Amazon EKS Pod Identity Agent*. Retrieved August 6, 2026, from https://docs.aws.amazon.com/eks/latest/userguide/pod-id-agent-setup.html
Expand Down
133 changes: 132 additions & 1 deletion scripts/ci/run_mv3_compatibility.py
Original file line number Diff line number Diff line change
Expand Up @@ -502,6 +502,102 @@ def _wait_for_linux_process_identity_exit(
time.sleep(min(0.05, remaining_seconds))


def _read_linux_process_identity_set(
process_ids: tuple[int, ...],
*,
required_root_identity: tuple[int, int],
) -> tuple[tuple[tuple[int, int], ...], int]:
"""Bind live sampled PIDs while explicitly accounting for already-exited descendants."""

if not process_ids or len(process_ids) > MAX_BROWSER_PROCESS_TREE_SIZE:
raise ValueError("invalid Linux process identity-set size")
if len(set(process_ids)) != len(process_ids):
raise ValueError("Linux process identity-set PIDs must be unique")
if not isinstance(required_root_identity, tuple) or len(required_root_identity) != 2:
raise ValueError("invalid Linux root process identity")
root_process_id, root_start_time_ticks = required_root_identity
if (
isinstance(root_process_id, bool)
or not isinstance(root_process_id, int)
or root_process_id <= 0
or isinstance(root_start_time_ticks, bool)
or not isinstance(root_start_time_ticks, int)
or root_start_time_ticks <= 0
):
raise ValueError("invalid Linux root process identity")
if process_ids[0] != root_process_id:
raise ValueError("Linux process identity set must start with the required root PID")

identities: list[tuple[int, int]] = []
pre_shutdown_exit_count = 0
for index, process_id in enumerate(process_ids):
if isinstance(process_id, bool) or not isinstance(process_id, int) or process_id <= 0:
raise ValueError("invalid Linux process identifier")
identity = _read_linux_proc_stat_process_identity(process_id)
if index == 0:
if identity is None:
raise RuntimeError("Linux Chromium root process identity disappeared before shutdown capture")
if identity != required_root_identity:
raise RuntimeError("Linux Chromium root process identity changed before shutdown capture")
identities.append(identity)
continue
if identity is None:
pre_shutdown_exit_count += 1
continue
identities.append(identity)
return tuple(identities), pre_shutdown_exit_count
Comment thread
seonghobae marked this conversation as resolved.


def _wait_for_linux_process_identity_set_exit(
process_identities: tuple[tuple[int, int], ...],
*,
timeout_seconds: float = PROCESS_EXIT_TIMEOUT_SECONDS,
) -> bool:
"""Wait under one shared deadline for every exact sampled process identity to exit."""

if not process_identities or len(process_identities) > MAX_BROWSER_PROCESS_TREE_SIZE:
raise ValueError("invalid Linux process identity-set size")
process_ids: list[int] = []
expected: dict[int, tuple[int, int]] = {}
for identity in process_identities:
if not isinstance(identity, tuple) or len(identity) != 2:
raise ValueError("invalid Linux process identity")
process_id, start_time_ticks = identity
if isinstance(process_id, bool) or not isinstance(process_id, int) or process_id <= 0:
raise ValueError("invalid Linux process identifier")
if (
isinstance(start_time_ticks, bool)
or not isinstance(start_time_ticks, int)
or start_time_ticks <= 0
):
raise ValueError("invalid Linux process start time")
if process_id in expected:
raise ValueError("Linux process identity-set PIDs must be unique")
process_ids.append(process_id)
expected[process_id] = identity
if (
isinstance(timeout_seconds, bool)
or not isinstance(timeout_seconds, (int, float))
or timeout_seconds < 0
or not math.isfinite(timeout_seconds)
):
raise ValueError("invalid Linux process-set exit timeout")

deadline = time.monotonic() + float(timeout_seconds)
while True:
live_identity_found = False
for process_id in process_ids:
current_identity = _read_linux_proc_stat_process_identity(process_id)
if current_identity == expected[process_id]:
live_identity_found = True
if not live_identity_found:
return True
remaining_seconds = deadline - time.monotonic()
if remaining_seconds <= 0:
return False
time.sleep(min(0.05, remaining_seconds))


def _sample_linux_process_rss_bytes(process_id: int) -> int:
"""Read one attributed Linux process RSS through a bounded ``/proc`` status file."""

Expand Down Expand Up @@ -930,6 +1026,8 @@ def _run_agent_task_browser_pass(
session_id: str | None = None
browser_process_id: int | None = None
browser_process_start_time_ticks: int | None = None
chromium_process_identities: tuple[tuple[int, int], ...] | None = None
chromium_process_pre_shutdown_exit_count: int | None = None
browser_failure_type: str | None = None
result: dict[str, Any] | None = None
driver = subprocess.Popen(
Expand Down Expand Up @@ -1112,6 +1210,16 @@ def _run_agent_task_browser_pass(
browser_process_id,
process_evidence,
)
(
chromium_process_identities,
chromium_process_pre_shutdown_exit_count,
) = _read_linux_process_identity_set(
chromium_process_ids,
required_root_identity=(
browser_process_id,
browser_process_start_time_ticks,
),
)
browser_process_rss_bytes = _sample_linux_process_rss_bytes(browser_process_id)
chromium_process_set_rss_bytes = _sample_linux_process_set_rss_bytes(
chromium_process_ids,
Expand All @@ -1138,6 +1246,9 @@ def _run_agent_task_browser_pass(
"saved_credential_services_disabled": True,
"browser_process_rss_bytes": browser_process_rss_bytes,
"chromium_process_count": chromium_process_count,
"chromium_process_pre_shutdown_exit_count": (
chromium_process_pre_shutdown_exit_count
),
"chromium_process_set_rss_bytes": chromium_process_set_rss_bytes,
"semantic_observation_bytes": semantic_observation_bytes,
"action_latency_ms": action_latency_ms,
Expand Down Expand Up @@ -1177,9 +1288,19 @@ def _run_agent_task_browser_pass(
}
if result is None:
raise RuntimeError("Agent Task browser pass returned no result after shutdown")
if chromium_process_identities is None:
raise RuntimeError("Agent Task Chromium process identities were not captured")
if chromium_process_pre_shutdown_exit_count is None:
raise RuntimeError("Agent Task Chromium pre-shutdown exit count was not captured")
chromium_process_set_terminated = _wait_for_linux_process_identity_set_exit(
chromium_process_identities
)
if not browser_process_terminated:
raise RuntimeError("Agent Task browser process did not terminate")
if not chromium_process_set_terminated:
raise RuntimeError("Agent Task Chromium process set did not terminate")
Comment thread
seonghobae marked this conversation as resolved.
result["browser_process_terminated"] = True
result["chromium_process_set_terminated"] = True
return result


Expand Down Expand Up @@ -1267,7 +1388,11 @@ def _run_agent_task_trial(
"browser_process_rss_bytes": result["browser_process_rss_bytes"],
"browser_process_terminated": result["browser_process_terminated"],
"chromium_process_count": result["chromium_process_count"],
"chromium_process_pre_shutdown_exit_count": result[
"chromium_process_pre_shutdown_exit_count"
],
"chromium_process_set_rss_bytes": result["chromium_process_set_rss_bytes"],
"chromium_process_set_terminated": result["chromium_process_set_terminated"],
"semantic_observation_bytes": result["semantic_observation_bytes"],
"action_latency_ms": result["action_latency_ms"],
"task_duration_ms": result["task_duration_ms"],
Expand Down Expand Up @@ -1699,10 +1824,16 @@ def main() -> int:
and trial.get("extensions_disabled") is True
and trial.get("profile_cleaned") is True
and trial.get("browser_process_terminated") is True
and trial.get("chromium_process_set_terminated") is True
and isinstance(trial.get("browser_process_rss_bytes"), int)
and trial["browser_process_rss_bytes"] > 0
and isinstance(trial.get("chromium_process_count"), int)
and 0 < trial["chromium_process_count"] <= MAX_BROWSER_PROCESS_TREE_SIZE
and isinstance(trial.get("chromium_process_pre_shutdown_exit_count"), int)
and not isinstance(trial["chromium_process_pre_shutdown_exit_count"], bool)
and 0
<= trial["chromium_process_pre_shutdown_exit_count"]
< trial["chromium_process_count"]
Comment thread
seonghobae marked this conversation as resolved.
and isinstance(trial.get("chromium_process_set_rss_bytes"), int)
and trial["chromium_process_set_rss_bytes"] > 0
and isinstance(trial.get("semantic_observation_bytes"), int)
Expand Down Expand Up @@ -1800,4 +1931,4 @@ def main() -> int:


if __name__ == "__main__":
raise SystemExit(main())
raise SystemExit(main())
125 changes: 125 additions & 0 deletions tests/test_agent_task_process_set_termination_contract.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,125 @@
"""Contract for proving the controlled Agent Task Chromium process set terminates."""

from __future__ import annotations

import pathlib
import runpy
import unittest

ROOT = pathlib.Path(__file__).resolve().parents[1]
RUNNER = ROOT / "scripts" / "ci" / "run_mv3_compatibility.py"


class AgentTaskProcessSetTerminationContractTests(unittest.TestCase):
"""Keep descendant cleanup evidence bounded, PID-reuse-safe, and fail closed."""

def test_runner_exposes_bounded_process_set_identity_and_exit_helpers(self) -> None:
"""A sampled Chromium tree needs exact PID/start-time identities before shutdown."""

namespace = runpy.run_path(str(RUNNER), run_name="agent_task_process_set_termination")
for expected in (
"_read_linux_process_identity_set",
"_wait_for_linux_process_identity_set_exit",
):
with self.subTest(expected=expected):
self.assertIn(expected, namespace)

def test_process_identity_set_reader_preserves_root_and_tolerates_exited_children(self) -> None:
"""Short-lived descendants may exit after the snapshot, but root identity stays exact."""

namespace = runpy.run_path(str(RUNNER), run_name="agent_task_process_set_reader")
reader = namespace["_read_linux_process_identity_set"]
original_reader = reader.__globals__["_read_linux_proc_stat_process_identity"]
identities = {10: (10, 101), 20: (20, 202), 30: (30, 303)}
try:
reader.__globals__["_read_linux_proc_stat_process_identity"] = identities.get
self.assertEqual(
reader((10, 20, 30), required_root_identity=(10, 101)),
(((10, 101), (20, 202), (30, 303)), 0),
)

reader.__globals__["_read_linux_proc_stat_process_identity"] = (
lambda process_id: None if process_id == 20 else identities[process_id]
)
self.assertEqual(
reader((10, 20, 30), required_root_identity=(10, 101)),
(((10, 101), (30, 303)), 1),
)

reader.__globals__["_read_linux_proc_stat_process_identity"] = (
lambda process_id: None if process_id == 10 else identities[process_id]
)
with self.assertRaisesRegex(RuntimeError, "root process identity disappeared"):
reader((10, 20, 30), required_root_identity=(10, 101))

reader.__globals__["_read_linux_proc_stat_process_identity"] = identities.get
with self.assertRaisesRegex(RuntimeError, "root process identity changed"):
reader((10, 20, 30), required_root_identity=(10, 999))
finally:
reader.__globals__["_read_linux_proc_stat_process_identity"] = original_reader

for process_ids, root_identity in (
((), (10, 101)),
((10, 10), (10, 101)),
((10, 20), (20, 202)),
((10, 20), (10, 0)),
):
with self.subTest(process_ids=process_ids, root_identity=root_identity):
with self.assertRaises(ValueError):
reader(process_ids, required_root_identity=root_identity)

def test_process_set_exit_waiter_uses_one_deadline_and_detects_any_live_identity(self) -> None:
"""A reused PID is exited evidence, but any exact surviving identity fails closed."""

namespace = runpy.run_path(str(RUNNER), run_name="agent_task_process_set_exit_waiter")
waiter = namespace["_wait_for_linux_process_identity_set_exit"]
original_reader = waiter.__globals__["_read_linux_proc_stat_process_identity"]
identities = ((10, 101), (20, 202), (30, 303))
try:
waiter.__globals__["_read_linux_proc_stat_process_identity"] = (
lambda _process_id: None
)
self.assertTrue(waiter(identities, timeout_seconds=0.0))

waiter.__globals__["_read_linux_proc_stat_process_identity"] = (
lambda process_id: (process_id, {10: 111, 20: 222, 30: 333}[process_id])
)
self.assertTrue(waiter(identities, timeout_seconds=0.0))

waiter.__globals__["_read_linux_proc_stat_process_identity"] = (
lambda process_id: (20, 202) if process_id == 20 else None
)
self.assertFalse(waiter(identities, timeout_seconds=0.0))
finally:
waiter.__globals__["_read_linux_proc_stat_process_identity"] = original_reader

for process_identities, timeout_seconds in (
((), 0.0),
(((10, 101), (10, 102)), 0.0),
(((10, 101),), -0.1),
):
with self.subTest(
process_identities=process_identities,
timeout_seconds=timeout_seconds,
):
with self.assertRaises(ValueError):
waiter(process_identities, timeout_seconds=timeout_seconds)

def test_successful_agent_task_requires_entire_sampled_process_set_to_terminate(self) -> None:
"""Successful acceptance must preserve already-exited descendants as explicit evidence."""

runner = RUNNER.read_text(encoding="utf-8")
for expected in (
"chromium_process_identities",
"chromium_process_pre_shutdown_exit_count",
'"chromium_process_set_terminated"',
'result["chromium_process_set_terminated"]',
'trial.get("chromium_process_set_terminated") is True',
"Agent Task Chromium process set did not terminate",
):
with self.subTest(expected=expected):
self.assertIn(expected, runner)


if __name__ == "__main__":
unittest.main()
Loading