Skip to content

[feat] Extend the delivery check to rules, agents and MCP #624

Closed

Description

@SaulMoro

Problem

#598 phase 3 gave doctor its first checks that look at the payload rather than the plumbing, and #625 implements them for two of the seven resource primitives:

skills   ✅ Skills delivered to <tool>     one destination per tool
docs     ✅ Team docs delivered            one destination, tree compare
env      ✅ (pre-existing env check)
hooks    ✅ (pre-existing hook check)
rules    ❌ nothing verifies what landed
agents   ❌ nothing verifies what landed
mcp      ❌ nothing verifies what landed

The three that are left are the same class of bug the issue opens with: the command reports success, the tool receives nothing. #598 said "skills first, because the destination resolver already exists" and predicted rules and agents would be "the same check with a different toolPath field". Implementing it showed that prediction is wrong in two places.

Why it is not just another toolPath field

Rules need their own destination resolver. The filename and the content change per tool:

rules/coding-style.md  (team repo)
  ├─ claude   → .claude/rules/coding-style.md          verbatim copy
  ├─ cursor   → .cursor/rules/coding-style.mdc         derived frontmatter
  └─ copilot  → .github/instructions/coding-style.md   rewritten body

ruleFileExtensionForTool and the two rewriters already exist (src/resources/rules.ts:195-215); nothing exposes "where would this rule land for this tool" as a read-only answer.

Agents break the "list × tools" shape. A skill is desired for every enabled tool. An agent carries spec.targets, so the desired set is a relation, not a product:

 skills:  desired[]        × tools[]        → every pair must exist
-agents:  desired[]        × tools[]
+agents:  desired[].targets                 → per item, and rendered per format

Plus each agent renders into one of three formats (renderForTool, .md / .toml / .json), so validity is format-dependent.

MCP is a different shape entirely, closer to the existing hook check: servers land as an entry inside each tool's native config file, not as a file of their own.

Proposed shape

Rather than teaching doctor about each resource type, give the handlers the seam and let one generic check loop them:

 abstract class ResourceHandler {
   abstract scanTeamForPull(...): Promise<ResourceItem[]>
   abstract pullItem(...): Promise<void>
+  /** Where `item` lands for each tool that can receive it. Read-only. */
+  deliveryTargets(teamConfig, localConfig, item): Promise<{ tool, dest }[]>
 }
flowchart LR
  subgraph today["today (#598)"]
    D1[buildChecks] --> S1["skillTargetForTool<br>doctor knows skills"]
    D1 --> S2["resolveDocsDestination<br>doctor knows docs"]
  end
  subgraph proposed
    D2[buildChecks] --> G["one delivery check"]
    G --> H1[SkillsHandler.deliveryTargets]
    G --> H2[RulesHandler.deliveryTargets]
    G --> H3[AgentsHandler.deliveryTargets]
    G --> H4[DocsHandler.deliveryTargets]
  end
Loading

skillTargetForTool (src/resources/skills.ts) and resolveDocsDestination (src/resources/docs.ts) are already that function under another name. This turns them into the first two implementations of a contract, and pullItem keeps using them so the write path and the check can never disagree.

The open decision

These checks now run at the end of every interactive teamai pull, not only in doctor. Skills and docs cost a stat per item. Rules and agents do not:

cost per pull
skills stat per skill per tool
docs stat per file in the bundle
rules read + stat per rule per tool (.mdc needs its frontmatter parsed)
agents parse every agent YAML to learn its targets and render extension

Since #625 that cost has a ceiling. The post-pull pass runs under a 5 second budget that covers building the registry as well as running it, and going over is all or nothing:

teamai pull
  buildChecks + runChecks
    within 5s  ->  print each failure with its fix
    over 5s    ->  "Post-pull checks did not run."
                   the member gets no check at all, not even the cheap ones

An agents check that parses every YAML therefore spends the budget the skills and docs checks need, and those are the ones that catch the bug this work started from. The pass also skips a scope another process has locked, so the concurrent case pays nothing either way.

The question to settle before implementing is whether rules and agents run in the post-pull pass at all, or only in teamai doctor. Check is where that would be expressed, and it now carries two independent axes rather than the one #598 left:

interface Check {
  source: 'local' | 'provider'   // #598: where the answer comes from
  reportedByPull?: string        // #625: the pull already said this itself
  // a third one for cost?
}

Three optional flags on one object is the point where it stops reading. Worth deciding whether cost is a third axis, another value on source, or something the registry says by building two lists instead.

Scope

  • ResourceHandler.deliveryTargets seam, with skills and docs migrated onto it, including the installed gate it subsumes: doctor and the sync answer "can this tool receive X" three different ways today (see comment below)
  • rules delivery check (per tool, per extension, .mdc frontmatter validity)
  • agents delivery check (spec.targets relation, three render formats)
  • mcp check in the hook-check shape (entry present in each tool's native config)
  • decide which of these run post-pull vs doctor-only

Related: #598, #574, #525, #342, #372.

Activity

  1. SaulMoro commented on Sep 18, 2026

    @SaulMoro
    CollaboratorAuthor

    Implementing the skills half in #625 turned up a seam this issue will need, so recording it here rather than as its own issue: it has no user-visible symptom left, and whoever extends the check to rules and agents is the person who will want it.

    What happened

    doctor's <tool> is installed probed the tool root, while skill delivery asks skillTargetForTool. Those disagree for OpenClaw, which keeps its skills under a workspace directory rather than under ~/.openclaw. A tool root on its own passed the check while delivery skipped the tool and its delivery check vanished with it, which is the failure this whole line of work exists to catch, reproduced inside the command written to catch it.

    Fixed in #625 by having the probe ask the same resolver the sync uses. That fix needed this:

    const INSTALL_PROBE_SKILL = '__teamai_probe__';
    
    export async function skillsReachTool(tool, configuredSkillsPath, localConfig): Promise<boolean> {
      return await skillTargetForTool(tool, configuredSkillsPath, localConfig, INSTALL_PROBE_SKILL) !== null;
    }

    A yes/no question has to invent a skill name, because skillTargetForTool fuses two questions: whether the tool can receive skills at all, and where a given skill lands. Only a comment says the name cannot affect the first one.

    Why it matters here

    Rules and agents each need the same gate, and today the codebase answers it three different ways for skills alone:

    skillTargetForTool            openclaw -> workspace dir, hermes -> hermes home,
      (src/resources/skills.ts)   else isToolInstalledForConfig
    isToolInstalledForConfig      copilot -> enabledAgents, else tool root
      (src/resources/base.ts)     used by agents.ts:420,501,559 and pull.ts:550,602
    ResourceHandler.isToolInstalled   tool root only, no copilot, no openclaw, no hermes
      (pull.ts:432)               inactive-skill cleanup, against resolveBaseDir
    

    So agents already answers the question with the generic probe, which is the spelling that was wrong for OpenClaw's skills. Adding a delivery check for agents on top of that would bake the same disagreement in, one resource type further along. The third spelling at pull.ts:432 judges installation for the inactive-skill cleanup differently again, and for OpenClaw it looks under a directory delivery never writes to.

    Suggested shape

    Correcting myself: this is not a separate proposal. deliveryTargets(teamConfig, localConfig, item) in the issue body already answers "which tools receive this item", so the gate below is that contract restricted to a yes/no, and the two should land as one piece of work rather than compete.

    Split the gate from the path, so the gate can be asked on its own and a new caller cannot get it wrong by omission:

    +/** Whether `tool` receives this resource type at all on this machine. */
    +export async function toolReceives(
    +  type: ResourceType,
    +  tool: string,
    +  configuredPath: string | undefined,
    +  localConfig: LocalConfig,
    +): Promise<boolean>
    +
     export async function skillTargetForTool(tool, configuredPath, localConfig, skillName, sourcePath?) {
    -  if (!configuredPath) return null;
    -  if (tool === 'openclaw') { ... }
    -  if (tool === 'hermes') { ... }
    -  if (!await isToolInstalledForConfig(tool, configuredPath, localConfig)) return null;
    +  if (!await toolReceives('skills', tool, configuredPath, localConfig)) return null;
       ...
     }

    Then doctor's installed check, the delivery checks for skills, rules and agents, the cleanup at pull.ts:432, and the agents gates all read one function, and skillsReachTool and its invented probe name go away. If deliveryTargets lands first, toolReceives is one line on top of it and does not need to exist as its own function.

    Worth doing before the rules and agents checks rather than after: added afterwards it is a change across five call sites plus three new ones, and each of those is a place where the check and the sync can drift apart again.

    Related: #625 also left resolveSkillDestination with two behaviours depending on whether sourcePath is passed, held together by a comment. That one is narrower and can ride along with this.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions