Skip to content

Polish README for submission #6

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
codeswithroh merged 1 commit into from
Jul 21, 2026
Merged

Polish README for submission #6

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
159 changes: 120 additions & 39 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,71 +1,134 @@
# TaskTape
# TaskTape Replay

**Turn a narrated bug recording into a living regression check.**
> Turn an agent-reproduced bug into a replayable regression check with local evidence.

TaskTape Replay is a macOS desktop tool for teaching repeatable regression checks. Record the bug, explain the expected result, and TaskTape builds an editable check that can be run again after the product changes.
[![CI](https://github.com/codeswithroh/tasktape/actions/workflows/ci.yml/badge.svg)](https://github.com/codeswithroh/tasktape/actions/workflows/ci.yml)
[![Release](https://img.shields.io/github/v/release/codeswithroh/tasktape?label=release)](https://github.com/codeswithroh/tasktape/releases/latest)
[![License: MIT](https://img.shields.io/badge/license-MIT-0f766e.svg)](LICENSE)
[![Platform](https://img.shields.io/badge/platform-macOS%20Apple%20Silicon-111827.svg)](https://tasktape-replay.vercel.app/)

Claude Code and Codex can also operate TaskTape directly through its built-in local MCP server. An agent can launch an instrumented browser, reproduce a bug, collect screenshots, DOM snapshots, console messages, network failures, and an action trace, then compile that evidence into the same Replay check a human can review and run.
TaskTape Replay is a macOS desktop app that lets Claude Code or Codex reproduce a local browser bug through MCP, capture the evidence, and turn that session into a check you can run again.

## Current status
It records actions, screenshots, DOM snapshots, console logs, network failures, and Playwright traces. GPT-5.6 can replay the workflow on the real interface, evaluate the final screen, and save a clear passed, failed, or inconclusive result.

The Build Week reference flow records a browser bug, captures the expected outcome by voice or text, replays the interaction with OpenAI computer use, and judges the final screen against that outcome. Each run stores a passed, failed, or inconclusive result with the final screenshot and a concise evidence trail.
![TaskTape Replay showing an agent-created check with evidence, report, and Playwright export controls](site/assets/tasktape-portable-check.png)

The agent-operated reference flow is limited to local HTTP and HTTPS development servers. TaskTape creates a headed isolated Chrome session and exposes 12 bounded MCP tools for observation, interaction, evidence capture, check creation, and explicit execution. Finishing a capture never runs or schedules the check.
## Contents

Agent-created checks keep their evidence attached. Users can inspect screenshot, console, network, and trace counts, copy a ticket-ready Markdown report, reveal the local files, or export the observed steps as an ordinary Playwright TypeScript test. Exported tests use no LLM at run time and remain reviewable before they enter source control or CI.
- [Why TaskTape](#why-tasktape)
- [What It Does](#what-it-does)
- [Download](#download)
- [Quick Start](#quick-start)
- [Agent Connection](#agent-connection)
- [Architecture](#architecture)
- [Verification](#verification)
- [Limitations](#limitations)
- [Development](#development)

Checks have editable instructions and can run now or on hourly, daily, weekday, or weekly timing. Manual and scheduled results appear in Run history, so a team can see when a previously fixed behavior regresses.
## Why TaskTape

A spoken schedule such as "every Monday at 9 AM" prefills the save controls. It never becomes active until the user confirms unattended execution and saves the task. The Scheduled view shows the next run, last result, pause or resume, and Run now controls.
Bug reports often arrive as screen recordings. That still leaves an engineer to reproduce the issue, inspect logs, write a regression test, and remember to run it later.

Recording starts with TaskTape's visual source gallery, which shows full displays and every currently available application window as named thumbnail tiles. The selected source is revalidated in Electron's main process immediately before capture.
TaskTape keeps the useful part of that debugging session. An agent can reproduce the bug once, TaskTape stores the evidence, and the result becomes a reusable check with history, scheduling, and exportable Playwright code.

Users can add or replace their own OpenAI API key from Settings. App-managed keys are encrypted through the operating system's secure storage and never exposed back to the renderer; `.env.local` remains a development-only fallback.
## What It Does

The initial release is macOS-first and is being built for OpenAI Build Week. The hackathon demo proves one complete browser regression loop deeply. It does not claim broad, deterministic automation across every desktop application.
| Capability | What happens |
| ---------------------- | ------------------------------------------------------------------------------------- |
| Agent-operated capture | Claude Code or Codex controls a local browser through TaskTape's MCP server. |
| Evidence bundle | TaskTape stores actions, screenshots, DOM, console, network, and trace files locally. |
| GPT-5.6 replay | OpenAI computer use replays the saved workflow against the current interface. |
| Visual verdict | GPT-5.6 evaluates the final screen against the expected outcome. |
| Portable handoff | Export a ticket-ready Markdown report or ordinary Playwright TypeScript test. |
| Scheduling | Run checks manually or on hourly, daily, weekday, or weekly timing. |

## Download

Download the Apple Silicon beta from the [TaskTape Replay site](https://tasktape-replay.vercel.app/) or from [GitHub Releases](https://github.com/codeswithroh/tasktape/releases/latest). The DMG contains TaskTape's tested browser runtime, so Google Chrome is not required.
Download the Apple Silicon beta:

The beta is not signed or notarized yet. After copying TaskTape to Applications, macOS may require you to control-click the app, choose Open, and confirm once. A signed release requires an Apple Developer ID certificate and notarization credentials.
- [TaskTape Replay site](https://tasktape-replay.vercel.app/)
- [GitHub Releases](https://github.com/codeswithroh/tasktape/releases/latest)
- [Direct DMG download](https://github.com/codeswithroh/tasktape/releases/latest/download/TaskTape-latest-arm64.dmg)

The reliable local-file capability learns extension groups and child-folder destinations from the demonstration, then organizes top-level files inside one approved folder. The user does not configure separate video or image fields. Tasks that need an application or website use OpenAI computer use through a native macOS input adapter. Model safety checks stop the run for review, and the agent is bounded to 25 turns. Scheduled runs require TaskTape to be open and the Mac to be awake. Background launch, history search, rollback, and safety-check resumption are not yet implemented.
The DMG includes TaskTape's tested browser runtime, so Google Chrome is not required.

## Product loop
> The beta is not Apple Developer ID signed or notarized yet. macOS may ask you to control-click the app, choose Open, and approve it once.

1. Record the bug yourself or ask Claude Code or Codex to reproduce it through TaskTape.
2. Capture the expected result by voice, text, or the MCP session request.
3. Review the learned replay steps, trace evidence, and success condition.
4. Save the check and choose whether it should run on a schedule.
5. Replay it against the current build.
6. Inspect the verdict, final screenshot, evidence, and run history.
## Quick Start

## Agent connection
1. Open TaskTape.
2. Go to Settings.
3. Copy the MCP command for Claude Code or Codex.
4. Ask the agent to reproduce a bug on a local development URL.
5. Review the generated check in TaskTape.
6. Run it, schedule it, copy the report, or export Playwright.

TaskTape shows the current local endpoint and copyable setup commands in Settings. With the app open, the default commands are:
## Agent Connection

TaskTape shows the active local endpoint in Settings. With the app open, the default commands are:

```bash
claude mcp add --transport http tasktape http://127.0.0.1:19790/mcp
codex mcp add tasktape --url http://127.0.0.1:19790/mcp
```

Ask the connected agent to reproduce a bug on a local development URL and turn it into a Replay check. Agent evidence is stored under TaskTape's local application data, never in the repository.
Then ask your agent something like:

## Documentation
```text
Use TaskTape to reproduce this local browser bug and turn it into a Replay check.
The selected category should still be Video after saving.
```

- [Product brief](docs/product-brief.md)
- [Architecture](docs/architecture.md)
- [Agent-operated replay milestone](docs/agent-mcp-milestone.md)
- [Voice and scheduling research](docs/voice-and-scheduling-research.md)
- [Reddit competitor research](docs/reddit-market-research.md)
- [Market validation](docs/market-validation.md)
- [Milestone roadmap](docs/roadmap.md)
- [Verification log](docs/verification.md)
Agent evidence is stored under TaskTape's local app data, never in the repository.

## Architecture

```mermaid
flowchart LR
Agent["Claude Code or Codex"] --> MCP["TaskTape MCP server"]
MCP --> Browser["Instrumented local browser"]
Browser --> Evidence["Actions, DOM, screenshots, logs, trace"]
Evidence --> Check["Reviewable Replay check"]
Check --> Replay["GPT-5.6 computer-use replay"]
Replay --> Verdict["Passed, failed, or inconclusive result"]
Check --> Export["Markdown report and Playwright export"]
```

TaskTape is built with Electron, React, TypeScript, Vite, MCP, Playwright, Vitest, Zod, nut.js, and the OpenAI API.

## Verification

The current release was verified with:

- `pnpm check`: formatting, lint, TypeScript, 66 tests, and production build.
- `pnpm test:e2e`: 10 Electron journeys.
- `pnpm test:site`: desktop and mobile landing-page checks.
- `pnpm package:mac`: production DMG and ZIP build.
- Mounted-DMG MCP test: app launched from the read-only DMG and completed the agent check flow.
- Public DMG verification: matching SHA-256 and valid `hdiutil verify`.

Full evidence is recorded in [docs/verification.md](docs/verification.md).

## Limitations

TaskTape is macOS-first. The Build Week demo proves one complete browser regression loop deeply rather than claiming broad automation across every desktop app.

Current boundaries:

- Agent-operated Replay is limited to local HTTP and HTTPS development servers.
- Scheduled runs require TaskTape to stay open and the Mac to stay awake.
- The macOS beta is ad-hoc signed, but not notarized.
- General desktop automation is bounded by model safety checks, 25 replay turns, and explicit user review.
- Background launch, rollback, history search, direct issue-tracker OAuth, and cloud execution are not implemented yet.

## Development

Prerequisites: Node.js 22+, pnpm 11.7.0, Xcode Command Line Tools, and macOS for desktop capture and computer-control verification. The packaging command downloads the pinned Playwright Chromium runtime into the release bundle.
Prerequisites:

- macOS
- Node.js 22+
- pnpm 11.7.0
- Xcode Command Line Tools

```bash
cd /Users/rohitpurkait/Documents/codex_build_week
Expand All @@ -74,12 +137,30 @@ pnpm check
pnpm dev
```

The separately invoked live model test regenerates its screenshot fixture before calling OpenAI:
Package the macOS app:

```bash
pnpm package:mac
```

Run the live model gate only when you intend to spend API credits:

```bash
pnpm test:live
```

The ignored `.env.local` file contains local credentials. Start from `.env.example` on a new machine and never commit secret values.
Local credentials belong in `.env.local`. Start from [.env.example](.env.example) on a new machine and never commit secrets.

## Documentation

- [Product brief](docs/product-brief.md)
- [Architecture notes](docs/architecture.md)
- [Agent-operated replay milestone](docs/agent-mcp-milestone.md)
- [Reddit competitor research](docs/reddit-market-research.md)
- [Market validation](docs/market-validation.md)
- [Milestone roadmap](docs/roadmap.md)
- [Verification log](docs/verification.md)

## License

See `AGENTS.md` for the engineering and verification rules that govern this repository.
TaskTape Replay is released under the [MIT License](LICENSE).
Loading