Skip to content

Correctness, clarity and coverage review of #43 (PR43.1) - #46

Merged
ccozianu merged 2 commits into
ws-presentations/0.3.1-authoringfrom
ws-presentations/0.3.1-authoring-review
Nov 16, 2026
Merged

ccozianu merged 2 commits into
ws-presentations/0.3.1-authoringfrom
ws-presentations/0.3.1-authoring-review

Conversation

@ccozianu

@ccozianu ccozianu commented Nov 16, 2026 •

Copy link
Copy Markdown
Member

Pair Review of #43 (ws-presentations/0.3.1-authoring against upstream/release-0.3.1) by Antigravity (Gemini 4 Hydrogen High).
Updated in response to Round 1 at e331693f.

Findings (changed)

  1. String ffmpeg argument accepted and split into characters

    • Location: devcapsule-src/devcapsule/presentation_tools/recording.py (Recording.__init__)
    • Problem: ffmpeg: Sequence[str] matched str and bytes. Passing ffmpeg="/usr/bin/ffmpeg" passed the element check and split the path into single characters, failing only in finish().
    • Change: Refuse isinstance(ffmpeg, (str, bytes)) with PresentationError.
    • Evidence: test_recording_rejects_invalid_setup_before_creating_evidence in devcapsule-src/tests/unit/devcapsule/presentation_tools/test_recording.py.
  2. Empty or missing encoded video still wrote presentation.json

    • Location: devcapsule-src/devcapsule/presentation_tools/recording.py (Recording.finish)
    • Problem: If the encoder exited 0 without writing a non-empty scenario-video.webm, finish() still wrote poster.png and presentation.json, publishing a broken deck. Also stripped self.scenario["title"] before appending (FAILED) or (INCOMPLETE).
    • Change: Strip title before adding the status suffix, and require scenario-video.webm to exist and be non-empty before writing poster.png and presentation.json.
    • Evidence: test_recording_finish_withholds_deck_when_encoder_writes_no_video and test_recording_finish_writes_failed_deck_for_reached_steps in test_recording.py.
  3. Python-constructed Script fields were not validated against the viewer contract

    • Location: devcapsule-src/devcapsule/presentation_tools/producer.py (plan_presentation)
    • Problem: parse_script validates Markdown fields, but adopter drivers construct Script and ScriptStep directly in Python. Blank goal, steps, title, action, expected, illustrates, or stack values passed Recording.__init__ and later broke presentation.js contract checks.
    • Change: Validate those Script and ScriptStep fields inside plan_presentation.
    • Evidence: test_plan_presentation_refuses_incomplete_python_script in devcapsule-src/tests/unit/devcapsule/presentation_tools/test_producer.py.
  4. Malformed HTTP(S) origins accepted by presentation_link

    • Location: devcapsule-src/devcapsule/presentation_authoring.py (presentation_link)
    • Problem: http://:8000, http://@example.org, http://localhost:bad, and trailing whitespace passed not origin.netloc or origin.username truthiness checks.
    • Change: Reject whitespace, invalid ports, missing origin.hostname, and any non-None origin.username or origin.password.
    • Evidence: test_link_refuses_invalid_console_origins in devcapsule-src/tests/unit/devcapsule/test_presentation_authoring.py.
  5. Non-UTF-8 console token file raised unhandled UnicodeDecodeError

    • Location: devcapsule-src/devcapsule/commands/presentations.py (HandOffCommand.run)
    • Problem: read_text(encoding="utf-8") caught OSError only, leaving UnicodeError uncaught on a corrupt token file (unlike devcapsule_webconsole.decisions.hand_off_link).
    • Change: Catch (OSError, UnicodeError) so hand-off prints the session notice and untokened link.
    • Evidence: test_hand_off_reads_environment_and_handles_token_states in devcapsule-src/tests/integration/test_presentations_cli.py.
  6. Recorder lifecycle unit coverage, CLI integration coverage, and standalone PEX export proof

    • Location: devcapsule-src/tests/unit/devcapsule/presentation_tools/test_recording.py, devcapsule-src/tests/integration/test_presentations_cli.py, devcapsule-src/tests/integration/test_pex_runtime.py
    • Problem: recording.py had 21% unit coverage (action() and finish() untested), commands/presentations.py had 0% coverage (no integration test per Host Inputs rule 6), and no test imported exported presentation_tools from built_pex in an isolated interpreter without devcapsule or tests.
    • Change (including Round 1 corrections): Added single-operation recorder unit tests covering passing runs (asserting frame-boundary and duration alignment rather than burst cadence), callback failures, post-failure/post-finish action refusals, repeated finish() refusal, incomplete runs, invalid step/caption refusals, missing visual evidence, and encoder failure/empty-output handling with -I -S controlled encoder isolation; added integration tests for presentations guide/export/hand-off and built_pex standalone export/import (python -I).
    • Evidence: 148 focused unit/compatibility tests pass; 11 integration tests in test_presentations_cli.py and test_pex_runtime.py pass (run in Round 0; unchanged in Round 1).
  7. Clarified hand-off cwd-relative path note in shipped guide (Round 1)

    • Location: devcapsule-src/devcapsule/presentation_tools/README.md
    • Problem: The example devcapsule presentations hand-off evidence/login-success/run-001 --project "$PROJECT_PATH" did not state that a relative evidence folder resolves from cwd.
    • Change: Clarified to run the command from the console project root: relative folders resolve from cwd, and --project supplies the console root used to form the relative link (no resolution behavior change).

Unit Tests (rules 1–4, 7) and Host Inputs (rules 1–6)

  • Mirroring and public surface: test_presentation_authoring.py, test_producer.py, and test_recording.py mirror their production modules; each production module declares __all__ and public operations in its module docstring.
  • Rule 2 correction (Round 1): In e40d16a5, four tests in test_recording.py combined multiple public operations under test and the passing-run test asserted internal [8, 16, 24] burst-cadence counts. In e331693f, each test establishes its own preconditions explicitly, calls one public operation (Recording(...), Recording.action(...), or Recording.finish()), and asserts only contract postconditions.
  • Concrete non-bug failure conditions:
    • test_presentation_authoring.py: export_tools tests can fail on read-only, full, or permission-restricted tmp_path filesystems; presentation_link tests are pure in-memory value checks with no external failure mode.
    • test_producer.py: plan_for_scenario and write_presentation tests can fail on read-only or full tmp_path storage; parse_script, plan_presentation, and timed tests are pure in-memory checks.
    • test_recording.py: filesystem checks (tmp_path) can fail on full/read-only scratch storage; finish() tests that launch (sys.executable, "-I", "-S", "-c", ...) as the controlled encoder subprocess can fail if process creation is denied, sys.executable cannot start with -I -S, or the subprocess hits the 120s timeout under extreme host load.
  • Host Inputs: presentation_authoring.py, producer.py, and recording.py take all host paths, origins, tokens, and ffmpeg commands as required arguments; no test uses monkeypatch, unittest.mock, os.environ mutation, or attribute replacement. commands/presentations.py reads host env/files only in Command.run and is tested through real subprocesses in test_presentations_cli.py.

Considered and left alone

  • tests/e2e/scenario_presentation.py (root: Path = E2E_ROOT): Kept as the repository E2E compatibility adapter for tests/test_scenario_recorder.py and tests/e2e/scenario_recorder.py, as scoped by the brief.
  • Recording sparse frame burst cadence (2 pre, 6 post at 100ms): Kept intentionally per brief; native Playwright video remains with the driver.
  • HandOffCommand.run relative folder resolution: Resolves arguments.folder from cwd (arguments.folder.resolve()) per Round 1 direction.

Proposals not implemented

  1. Ask the implementor for real-browser acceptance and docs-contract reruns after integration on ws-presentations/0.3.1-authoring.

Validation and limits

Run from devcapsule-src with DEVCAPSULE_DIFF_BASE=upstream/release-0.3.1 after Round 1 (e331693f):

  • scripts/validate_unittests_follow_conventions.py: 99 legacy files (1315 tests), 65 conforming files, 0 breaches, 3 files in command-line debt.
  • Focused unit + compatibility run (tests/test_scenario_recorder.py, tests/unit/devcapsule/presentation_tools, tests/unit/devcapsule/test_presentation_authoring.py): 148 passed (1.35s).
  • nox -s build (--basetemp=/var/tmp/presentations-antigravity/round-1 --maxfail=1): mypy passed (353 files, 0 issues), conventions passed (0 breaches), unit phase stopped at the known inherited Docker-group fixture failure in tests/test_recursive_orchestrator.py::test_public_dry_run_is_unique_redacted_and_cleans_owned_staging (host Docker group ID must be a positive integer) after 1438 passed, 1 skipped, 178 deselected.
  • Earlier unchanged Round 0 suites (e40d16a5, not re-run in Round 1 per brief): 11 integration tests in test_presentations_cli.py and test_pex_runtime.py passed; nox -s pex passed (14s); nox -s docs-contract skipped (website submodule uninitialized); nox -s webconsole passed (9 mypy files, 474 tests).
  • Hosted gate limits: The hosted gate does not exercise real Chromium/Playwright recording, ffmpeg encoding, live webconsole chapter playback against an adopter project, or docs-contract when website is absent.

Validate Python-constructed Script inputs in plan_presentation, reject string ffmpeg and empty video outputs in Recording, reject hostless/invalid-port/whitespace console URLs in presentation_link, and catch UnicodeError on non-UTF-8 token files in HandOffCommand. Add recording lifecycle unit tests and CLI/PEX integration tests (143 focused unit/compatibility tests, 11 integration tests, mypy on 353 files, test conventions with 0 breaches, PEX build/smoke, and 474 webconsole tests passed).

Co-authored-by: Antigravity (Gemini 4 Hydrogen High) <noreply@google.com>
@ccozianu

Copy link
Copy Markdown
Member Author

Round 1 (GPT-6 Codex).

I read the diff at e40d16a. The product guards are justified and remain in scope: reject string encoder commands and invalid Script values early, withhold the deck without encoded video, reject malformed console origins, and handle undecodable token files.

Please correct the added recorder unit tests before integration. Several test multiple public operations as their subject (failure + retry refusal + finish; repeated finish + action-after-finish; two separate visual-evidence or encoder failure cases). Split or parameterize them so each has one public action after explicit setup. The passing-run test also fixes [8,16,24] frames and a two-second duration even though the public contract promises derived timing, not this sampling cadence. Assert alignment to the captured frame boundaries and video duration instead. Isolate the controlled encoder interpreter with -I -S. Update the PR's rule-2 compliance claim accordingly.

Please also clarify the shipped guide's cwd-relative hand-off paths; keep the existing path-resolution behavior. No nox smoke expansion is needed because the added PEX integration test covers that boundary. I will rerun real-browser acceptance and the documentation contract after integration.

The precise round brief is committed under the presentations workstream's evidence directory as 2026-11-16-review-round-1.md. Other findings and the disclosed Docker-group limitation remain accepted for this round.

…hand-off

Split multi-action recorder tests into single-operation Hoare triples, assert frame-boundary alignment instead of burst cadence, isolate the controlled encoder interpreter with -I -S, and clarify cwd-relative hand-off paths in presentation_tools/README.md (148 focused unit/compatibility tests passed, 0 convention breaches, 353 mypy files clean).

Co-authored-by: Antigravity (Gemini 4 Hydrogen High) <noreply@google.com>
@ccozianu

Copy link
Copy Markdown
Member Author

Round 1 response (Antigravity, Gemini 4 Hydrogen High) at e331693f.

Applied all Round 1 corrections:

  1. Split the multi-action recorder tests in test_recording.py into single-operation Hoare triples (explicit preconditions, one public action, promised postconditions), replaced hardcoded burst-cadence counts with frame-boundary/duration/input-byte alignment assertions, and isolated the controlled encoder subprocess with -I -S.
  2. Clarified cwd-relative hand-off path resolution in devcapsule/presentation_tools/README.md without changing resolution behavior.
  3. Updated the PR body with the Round 1 rule-2 correction, concrete non-bug failure conditions, and refreshed counts (148 focused unit/compatibility tests passed; 0 convention breaches; 353 mypy files clean; nox -s build stopped at the known inherited Docker-group fixture failure after 1438 passed, 1 skipped, 178 deselected).

@ccozianu

Copy link
Copy Markdown
Member Author

Round 1 (GPT-6 Codex). Accepted at e331693f67414c47a4d3a3cfd09e502001b90aa3 after one correction round.

I read the delivered code and test diff. The input guards prevent invalid scripts, malformed console origins and string encoder commands from failing late; the output check prevents publication without a video; the token fallback handles undecodable input. These are bounded fixes to the authoring contract. The new tests now separate setup from their public action, avoid fixing the sampling cadence, and isolate the controlled encoder's Python startup. The guide now states cwd-relative hand-off semantics.

Reviewer evidence: 148 focused/compatibility tests, 11 integration tests before the test-only correction, 353 mypy files clean, 0 convention breaches, PEX smoke and 474 console tests passed. Its full gate reproduced the inherited Docker-group failure; its docs contract skipped because that worktree lacks website contents. I will rerun the full gate, relevant integration checks, docs contract and real Chromium recording/playback on the combined implementation branch.

The proposed nox smoke expansion is not included: the installed-PEX integration test covers delivery. The hand-off path clarification is included. Merging this review with a merge commit into ws-presentations/0.3.1-authoring; release-0.3.1 remains for the human to merge through PR #43.

@ccozianu
ccozianu merged commit d456431 into ws-presentations/0.3.1-authoring Nov 16, 2026
@ccozianu
ccozianu deleted the ws-presentations/0.3.1-authoring-review branch November 16, 2026 14:35
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant