Adding in the ability to add a git worktree into the created workspace for a test by richardpark-msft · Pull Request #154 · microsoft/waza

richardpark-msft · 2026-03-21T04:14:08Z

Adding in the ability to add a git worktree into a workspace created for a copilot run. This is most useful for people that are developing skills within a repo they're also using the skills - this lets them get a clean copy of the repo for each test run. See 'internal/models/testdata/git-resources-task-example.yaml' for an idea of what the YAML looks like to use it.

Future plans are to add in more git clone methods, but this is a nice cheap way of boostrapping a clean copy of a repo, when working locally.

We also now validate that the task configuration is valid before starting - prior to this we were apparently just throwing out log messages but no errors, so a user could start with a partially invalid configuration and not realize that critical checks, etc... weren't even being loaded.

Fixes #121

richardpark-msft · 2026-03-21T04:18:05Z

(just a note, the coding is done, we have tests in place, but I want to see if I can add some more E2E tests before moving this into final review)

…for a copilot run. This is most useful for people that are developing skills within a repo they're also using the skills - this lets them get a clean copy of the repo for each test run. See 'internal/models/testdata/git-resources-task-example.yaml' for an idea of what the YAML looks like to use it. Future plans are to add in more git clone methods, but this is a nice cheap way of boostrapping a clean copy of a repo, when working locally. We also now validate that the task configuration is valid before starting - prior to this we were apparently just throwing out log messages but no errors, so a user could start with a partially invalid configuration and not realize that critical checks, etc... weren't even being loaded.

Copilot

Pull request overview

This PR adds first-class “git resources” to task inputs so a test run can materialize a clean copy of a local repository (via git worktree) inside the per-run workspace, and tightens up task input validation so invalid file/resource config fails fast instead of only warning.

Changes:

Extend the task schema and models to support inputs.repos (git worktree resources) and stricter inputs.files validation.
Introduce ExecutableTestCase to pre-validate/materialize file and git resources before execution, and plumb git resources through to the execution engine(s).
Add execution-layer git worktree support plus tests and docs updates.

Reviewed changes

Copilot reviewed 22 out of 22 changed files in this pull request and generated 12 comments.

Show a summary per file

File	Description
`schemas/task.schema.json`	Adds `inputs.repos` and tightens `resourceRef` validation (needs `workdir` added + stricter gitResource def).
`internal/orchestration/runner.go`	Adds `ExecutableTestCase`, preloads resources, and passes git resources/workdir into execution requests.
`internal/orchestration/runner_test.go`	Updates runner unit tests to use `ExecutableTestCase`.
`internal/orchestration/runner_orchestration_test.go`	Updates resource-loading tests for new `(resources, gitResources, err)` signature.
`internal/orchestration/filter.go`	Updates filtering helpers to operate on `ExecutableTestCase`.
`internal/orchestration/filter_test.go`	Updates filter tests for `ExecutableTestCase`.
`internal/models/testcase.go`	Adds git resource types and `ResourceRef.Validate` (missing `WorkDir` field currently blocks compilation).
`internal/models/testcase_test.go`	Adds tests for git resources, workdir, and ResourceRef validation; refactors trigger YAMLs into files.
`internal/models/testdata/*`	Adds example task YAMLs for git resources, file resources, triggers, and workdir.
`internal/execution/engine.go`	Adds `GitResources` + `WorkDir` to `ExecutionRequest` and introduces `ResolveWorkDir`.
`internal/execution/copilot.go`	Plumbs git resources into workspace setup and adds cleanup tracking (but still doesn’t apply `WorkDir`).
`internal/execution/mock.go`	Adds git resource materialization/cleanup in mock engine (but changes `WorkspaceDir` semantics incorrectly).
`internal/execution/git.go`	Implements git worktree materialization and cleanup helpers.
`internal/execution/git_test.go`	Adds tests for git worktree behavior and workdir resolution (needs git author config for reliability).
`internal/execution/generate.go`	Adds mock generation for `GitResource`.
`internal/execution/execution_mocks.go`	Generated GoMock for `GitResource`.
`internal/execution/engine_shutdown_test.go`	Extends shutdown tests to ensure git resources are cleaned up.
`README.md`	Documents git resources + `workdir` usage (cleanup timing is currently misstated).

Comments suppressed due to low confidence (2)

schemas/task.schema.json:97

Task YAML/examples now use inputs.workdir, but the task schema doesn’t define a workdir property under inputs. Add it (as a string) so schema validation and editor tooling match the documented behavior.

      "properties": {
        "prompt": {
          "type": "string",
          "description": "The prompt message sent to the agent."
        },
        "context": {
          "type": "object",
          "description": "Arbitrary key-value metadata passed to the agent."
        },
        "files": {
          "type": "array",
          "items": {
            "$ref": "#/$defs/resourceRef"
          },
          "description": "File references or inline content provided to the agent."
        },
        "repos": {
          "type": "array",
          "items": {
            "$ref": "#/$defs/gitResource"
          },
          "description": "Git repositories provided to the agent workspace."
        },
        "environment": {
          "type": "object",
          "additionalProperties": {
            "type": "string"
          },
          "description": "Environment variables set during task execution."
        }

internal/models/testcase.go:33

TestStimulus is missing a WorkDir field, but the new YAML examples/tests (and runner.go via tc.Stimulus.WorkDir) rely on it. This will not compile and workdir: in task YAML will be ignored until a WorkDir string yaml:"workdir,omitempty"`` field is added to TestStimulus.

// TestStimulus defines the input for a test
type TestStimulus struct {
	Message     string            `yaml:"prompt" json:"message"`
	Metadata    map[string]any    `yaml:"context,omitempty" json:"metadata,omitempty"`
	Resources   []ResourceRef     `yaml:"files,omitempty" json:"resources,omitempty"`
	Repos       []GitResource     `yaml:"repos,omitempty" json:"repos,omitempty"`
	Environment map[string]string `yaml:"environment,omitempty" json:"environment,omitempty"`
}

Copilot · 2026-03-21T04:22:08Z

internal/models/testcase.go

+// GitResource specifies a git repository at a particular commit as a task input.
+// The repo is always auto-detected from the current working directory.
+type GitResource struct {
+	// Commit is the git commit we will start the git clone at
+	// Empty string just defaults to HEAD.
+	Commit string `yaml:"commit,omitempty" json:"commit,omitempty"`
+
+	// Type is the kind of git cloning we're doing.
+	Type GitType `yaml:"type" json:"type"`
+
+	// Source varies, depending on the type.
+	// - For 'worktree', Source is the folder where the git repository resides.
+	Source string `yaml:"source" json:"source"`
+


The GitResource doc comment says the repo is “auto-detected from the current working directory”, but the struct actually requires an explicit Source path. Please update the comment to match the new behavior to avoid confusing task authors.

This issue also appears on line 26 of the same file.

Copilot · 2026-03-21T04:22:08Z

schemas/task.schema.json

+    "gitResource": {
+      "type": "object",
+      "description": "A git repository checked out at a specific commit.",
+      "required": [
+        "type"
+      ],
+      "properties": {
+        "commit": {
+          "type": "string",
+          "description": "Git commit SHA, branch, or tag to check out. Defaults to HEAD."
+        },
+        "type": {
+          "type": "string",
+          "enum": [
+            "worktree"
+          ],
+          "description": "Git resource type. Currently only 'worktree' is supported."
+        },
+        "source": {
+          "type": "string",
+          "description": "Type-specific source location. For 'worktree', this is the folder where the git repository resides."
+        },
+        "dest": {
+          "type": "string",
+          "description": "Subdirectory name within the workspace. If omitted, the git repo becomes the workspace root."
+        }
+      },
+      "allOf": [
+        {
+          "if": {
+            "properties": {
+              "type": {
+                "const": "worktree"
+              }
+            },
+            "required": [
+              "type"
+            ]
+          },
+          "then": {
+            "required": [
+              "source"
+            ]
+          }
+        }
+      ]
    }


$defs.gitResource currently allows unknown keys because it lacks "additionalProperties": false (many other object defs in this schema are strict). Consider adding additionalProperties:false and (optionally) minLength: 1 for required strings like source to catch typos early.

Copilot · 2026-03-21T04:22:08Z

internal/execution/copilot.go

+	// Clean up worktrees before removing workspaces (worktrees may be inside workspace dirs)
+	for _, wt := range gitResources {
+		if err := wt.Cleanup(ctx); err != nil {
+			slog.Warn("failed to cleanup git resource", "error", err)
+		}
+	}


doShutdown cleans up git resources using the caller-provided ctx. If Shutdown is invoked with a canceled/deadline-exceeded context, git cleanup will be skipped/aborted, leaving stale worktrees behind. Consider using a background/best-effort context (or a fresh timeout) for cleanup operations.

Copilot · 2026-03-21T04:22:08Z

internal/execution/copilot.go

+	wts, err := CloneGitResources(ctx, gitResources, workspaceDir)
+	if err != nil {
+		return "", err
+	}
+	if len(wts) > 0 {
+		e.resourcesMu.Lock()
+		e.gitResources = append(e.gitResources, wts...)
+		e.resourcesMu.Unlock()
+	}


If CloneGitResources fails, setupWorkspace returns without cleaning up the newly created temp workspace (already appended to e.workspaces) and without cleaning any partially-created git resources. Consider cleaning up immediately on error (and/or have CloneGitResources rollback created resources) to avoid leaking temp dirs/worktrees when Execute fails early.

Copilot · 2026-03-21T04:22:09Z

internal/execution/mock.go

+	// Materialize git resources
+	wts, err := CloneGitResources(ctx, req.GitResources, m.workspace)
+	if err != nil {
+		return nil, fmt.Errorf("failed to materialize git resource in mock workspace: %w", err)
+	}
+	m.gitResources = append(m.gitResources, wts...)
+


m.gitResources is appended to on each Execute, but prior worktrees are never cleaned up when a new workspace replaces the old one (the old workspace dir is deleted earlier, but the source repo will still track the worktree). Consider cleaning up any existing m.gitResources before creating/materializing new git resources, and reset the slice per-execute.

Copilot · 2026-03-21T04:22:09Z

internal/execution/copilot.go


 	start := time.Now()

-	workspaceDir, err := e.setupWorkspace(req.Resources)
+	workspaceDir, err := e.setupWorkspace(ctx, req.Resources, req.GitResources)

 	if err != nil {
 		return nil, err


ExecutionRequest now carries WorkDir, but CopilotEngine doesn’t apply it anywhere (it only passes the workspace root into the session config). This means inputs.workdir will work in MockEngine but be ignored in real runs unless CopilotEngine.Execute resolves and uses it when setting the session working directory.

Copilot · 2026-03-21T04:22:09Z

internal/execution/mock.go

 		ToolCalls:    []models.ToolCall{},
 		Success:      true,
 		WorkspaceDir: m.workspace,
 	}

 	return resp, nil
 }

 func (m *MockEngine) Shutdown(ctx context.Context) error {


MockEngine sets ExecutionResponse.WorkspaceDir to the resolved workdir (agentCwd). WorkspaceDir is used by graders/tests as the workspace root; changing it breaks existing expectations (e.g., tests that read resp.WorkspaceDir/fixtures/...). Keep WorkspaceDir as the workspace root and, if needed, introduce a separate field for the agent CWD.

Copilot · 2026-03-21T04:22:09Z

internal/execution/git.go

+		targetDir := workspaceDir
+
+		if gitRes.Dest != "" {
+			targetDir = filepath.Join(workspaceDir, gitRes.Dest)


CloneGitResource joins workspaceDir with gitRes.Dest without validating that Dest stays within the workspace (e.g., dest: "../../.." would write outside the sandbox). Please validate Dest is relative/non-traversing (similar to ResolveWorkDir) before constructing targetDir.

Suggested change

targetDir = filepath.Join(workspaceDir, gitRes.Dest)

cleanDest := filepath.Clean(gitRes.Dest)

// Ensure destination is a relative, non-traversing path within the workspace.

if filepath.IsAbs(cleanDest) ||

cleanDest == ".." ||

strings.HasPrefix(cleanDest, ".."+string(os.PathSeparator)) {

return nil, fmt.Errorf("invalid git destination path %q", gitRes.Dest)

}

targetDir = filepath.Join(workspaceDir, cleanDest)

Copilot · 2026-03-21T04:22:10Z

internal/execution/git_test.go

+	_, err = runGitCommand(context.Background(), repoDir, "add", "hello.txt")
+	require.NoError(t, err)
+
+	_, err = runGitCommand(context.Background(), repoDir, "commit", "-m", "first and only file", "hello.txt")


mustCreateRepo runs git commit without configuring an author/committer identity. In clean CI environments without global git config, this often fails (“please tell me who you are”). Set GIT_AUTHOR_NAME/GIT_AUTHOR_EMAIL (and committer equivalents) for the test process or pass -c user.name=... -c user.email=... for the commit command to make the test reliable.

Suggested change

_, err = runGitCommand(context.Background(), repoDir, "commit", "-m", "first and only file", "hello.txt")

_, err = runGitCommand(

context.Background(),

repoDir,

"-c", "user.name=Waza Test",

"-c", "user.email=waza-test@example.com",

"commit", "-m", "first and only file", "hello.txt",

)

Copilot · 2026-03-21T04:22:10Z

README.md

+  dest: feature
+```
+
+Worktrees are automatically cleaned up after each task via `git worktree remove`.


The README says worktrees are cleaned up “after each task”, but the current implementation cleans up git resources during engine Shutdown (end of run). Please adjust the wording to match actual cleanup timing to avoid giving users the wrong expectation about when worktrees disappear.

Suggested change

Worktrees are automatically cleaned up after each task via `git worktree remove`.

Worktrees are automatically cleaned up at the end of the run (during engine shutdown) via `git worktree remove`.

richardpark-msft requested review from chlowell, spboyer and wbreza as code owners March 21, 2026 04:14

Copilot AI review requested due to automatic review settings March 21, 2026 04:14

richardpark-msft marked this pull request as draft March 21, 2026 04:14

Copilot started reviewing on behalf of richardpark-msft March 21, 2026 04:14 View session

richardpark-msft force-pushed the wz-git-worktree branch from bf022a3 to 99ec1b5 Compare March 21, 2026 04:19

Copilot AI reviewed Mar 21, 2026

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Adding in the ability to add a git worktree into the created workspace for a test#154

Adding in the ability to add a git worktree into the created workspace for a test#154
richardpark-msft wants to merge 1 commit intomicrosoft:mainfrom
richardpark-msft:wz-git-worktree

richardpark-msft commented Mar 21, 2026

Uh oh!

richardpark-msft commented Mar 21, 2026

Uh oh!

Copilot AI left a comment

Uh oh!

Copilot AI Mar 21, 2026

Uh oh!

Copilot AI Mar 21, 2026

Uh oh!

Copilot AI Mar 21, 2026

Uh oh!

Copilot AI Mar 21, 2026

Uh oh!

Copilot AI Mar 21, 2026

Uh oh!

Copilot AI Mar 21, 2026

Uh oh!

Copilot AI Mar 21, 2026

Uh oh!

Copilot AI Mar 21, 2026

Uh oh!

Copilot AI Mar 21, 2026

Uh oh!

Copilot AI Mar 21, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

-			targetDir = filepath.Join(workspaceDir, gitRes.Dest)
+			cleanDest := filepath.Clean(gitRes.Dest)
+			// Ensure destination is a relative, non-traversing path within the workspace.
+			if filepath.IsAbs(cleanDest) ||
+				cleanDest == ".." ||
+				strings.HasPrefix(cleanDest, ".."+string(os.PathSeparator)) {
+				return nil, fmt.Errorf("invalid git destination path %q", gitRes.Dest)
+			}
+			targetDir = filepath.Join(workspaceDir, cleanDest)

	Worktrees are automatically cleaned up after each task via `git worktree remove`.
	Worktrees are automatically cleaned up at the end of the run (during engine shutdown) via `git worktree remove`.

Conversation

richardpark-msft commented Mar 21, 2026

Uh oh!

richardpark-msft commented Mar 21, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Copilot AI Mar 21, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Mar 21, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Mar 21, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Mar 21, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Mar 21, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Mar 21, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Mar 21, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Mar 21, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Mar 21, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Mar 21, 2026

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants