Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
251 changes: 72 additions & 179 deletions src/mastra/agents/security-research/skill-prompt.ts

Large diffs are not rendered by default.

22 changes: 19 additions & 3 deletions src/mastra/agents/security-research/stage-agents.ts
Original file line number Diff line number Diff line change
Expand Up @@ -285,7 +285,14 @@ function stageModel(definition: StageDefinition) {
};
}

function stageInstructions(definition: StageDefinition) {
async function stageInstructions(
definition: StageDefinition,
capabilities: readonly string[] = [],
) {
const skillInstructions = await formatSecurityResearchSkillInstructions(
definition.skillHints,
{ capabilities },
);
return `${commonStageInstructions(definition.stage)}

Stage role:
Expand All @@ -296,7 +303,7 @@ ${definition.responsibilities.map((item) => `- ${item}`).join("\n")}

Relevant product skill instructions:
Product skills describe reusable runtime research procedures from sandbox/skills. Apply the relevant details below when they fit the assigned task, but keep the output scoped to this stage and the user's authorization.
${formatSecurityResearchSkillInstructions(definition.skillHints)}
${skillInstructions}

Output contract:
${definition.outputContract.map((item) => `- ${item}`).join("\n")}
Expand All @@ -309,7 +316,11 @@ function createStageAgent(definition: StageDefinition) {
id: definition.id,
name: definition.name,
description: definition.description,
instructions: stageInstructions(definition),
instructions: ({ requestContext }) =>
stageInstructions(
definition,
readRuntimeSkillCapabilities(requestContext.get("runtimeSkillCapabilities")),
),
model: stageModel(definition),
memory: createSecurityResearchStageMemory(
definition.stage,
Expand All @@ -334,6 +345,11 @@ function createStageAgent(definition: StageDefinition) {
});
}

function readRuntimeSkillCapabilities(value: unknown) {
if (!Array.isArray(value)) return [];
return value.filter((item): item is string => typeof item === "string");
}

export function buildSecurityResearchStageThreadId(
projectId: string,
stage: SecurityResearchStageId,
Expand Down
7 changes: 1 addition & 6 deletions src/mastra/config/editor.ts
Original file line number Diff line number Diff line change
Expand Up @@ -12,10 +12,6 @@ import { parse } from "llm-strings";
import { getDefaultModelFor } from "../../lib/models";
import { getSecurityResearchRenderPrompt } from "../../lib/render-json";
import { MARKDOWN_LITERAL_FORMATTING_INSTRUCTION } from "../../lib/security-chat/formatting-instructions";
import {
formatSecurityResearchSkillDirectory,
SECURITY_RESEARCH_SKILL_IDS,
} from "../agents/security-research/skill-prompt";

export const SECURITY_RESEARCH_PROMPT_BLOCK_IDS = {
operatingRules: "security-research-operating-rules",
Expand Down Expand Up @@ -128,8 +124,7 @@ Optimize prompt quality first, but keep cost, speed, and safety/auditability vis

For non-trivial work, turn the request into compact tasks with scope, expected evidence, approval boundary, stop condition, and useful output. Take the next safe approved action instead of waiting unnecessarily. Use an Observe -> Orient -> Decide -> Act loop after new evidence, failures, denials, or blocked work.

Search and load a full product procedure only when it is relevant. Available skills:
${formatSecurityResearchSkillDirectory(SECURITY_RESEARCH_SKILL_IDS)}
Use the product skill directory injected by the configured Mastra Workspace. Search and load a full product procedure only when it is relevant. Do not rely on a separate prompt-owned skill catalog.

Delegate focused stage-sized work when useful, preserve evidence and uncertainty in the final summary, and do not claim a finding without evidence. Treat authorization, approval, target scope, and non-destructive operation as hard constraints.
Make ordinary stage delegations self-contained and omit forked mode. Use a forked subagent only when explicitly necessary to continue from parent conversation context; it clones the parent thread and is not the normal delegation path.
Expand Down
256 changes: 218 additions & 38 deletions src/mastra/config/skill-search.ts
Original file line number Diff line number Diff line change
@@ -1,50 +1,230 @@
import {
BaseProcessor,
type ProcessInputStepArgs,
type ProcessInputStepResult,
SkillSearchProcessor,
BaseProcessor,
type ProcessInputStepArgs,
type ProcessInputStepResult,
} from "@mastra/core/processors";
import { createTool } from "@mastra/core/tools";
import { z } from "zod";

import { readSecurityResearchRuntimeToolProfile } from "../../lib/security-chat/runtime-tool-profile";
import { securityResearchWorkspace } from "./workspace";
import {
createProductSkillRegistry,
formatProductSkillDirectory,
type ProductSkillRegistry,
} from "../../server/skills/product-skill-registry";
import {
SECURITY_RESEARCH_SKILLS_ROOT,
securityResearchWorkspace,
} from "./workspace";

const readRequestContext = (requestContext: ProcessInputStepArgs["requestContext"], key: string) =>
requestContext?.get?.(key);
const readRequestContext = (
requestContext: ProcessInputStepArgs["requestContext"],
key: string,
) => requestContext?.get?.(key);

/** A foreground-command-only run deliberately exposes no overlapping discovery tools. */
export const shouldUseSecurityResearchSkillSearch = (
requestContext: ProcessInputStepArgs["requestContext"],
requestContext: ProcessInputStepArgs["requestContext"],
) =>
readSecurityResearchRuntimeToolProfile(
readRequestContext(requestContext, "runtimeToolProfile"),
) !== "foreground-command-only";

class RuntimeToolProfileSkillSearchProcessor extends BaseProcessor<"security-research-skill-search"> {
readonly id = "security-research-skill-search" as const;
readonly name = "Security Research Skill Search";

constructor(private readonly delegate: SkillSearchProcessor) {
super();
}

async processInputStep(args: ProcessInputStepArgs): Promise<ProcessInputStepResult> {
if (!shouldUseSecurityResearchSkillSearch(args.requestContext)) {
return { tools: args.tools };
}
return this.delegate.processInputStep(args);
}
readSecurityResearchRuntimeToolProfile(
readRequestContext(requestContext, "runtimeToolProfile"),
) !== "foreground-command-only";

type LoadedSkill = { revision: string; instructions: string };
type ThreadSkillState = {
snapshotKey: string;
loaded: Map<string, LoadedSkill>;
};

/**
* On-demand skill tools backed by the reviewed product registry. Search and load
* both re-read request-scoped capabilities and revalidate through Workspace.
*/
export class RegistryBackedSkillSearchProcessor extends BaseProcessor<"security-research-skill-search"> {
readonly id = "security-research-skill-search" as const;
readonly name = "Security Research Skill Search";
readonly providesSkillDiscovery = "on-demand" as const;

private readonly threadState = new Map<string, ThreadSkillState>();

constructor(private readonly registry: ProductSkillRegistry) {
super();
}

async processInputStep(
args: ProcessInputStepArgs,
): Promise<ProcessInputStepResult> {
if (!shouldUseSecurityResearchSkillSearch(args.requestContext)) {
return { tools: args.tools };
}

const capabilitiesForRequest = () =>
readRuntimeSkillCapabilities(
readRequestContext(args.requestContext, "runtimeSkillCapabilities"),
);
const capabilities = capabilitiesForRequest();
const snapshot = await this.registry.list({ capabilities });
const snapshotKey = [
snapshot.status,
snapshot.revision ?? "incomplete",
capabilityKey(capabilities),
].join(":");
const threadId = readThreadId(args);
let state = this.threadState.get(threadId);
if (!state || state.snapshotKey !== snapshotKey) {
state = { snapshotKey, loaded: new Map() };
this.threadState.set(threadId, state);
}

args.messageList.addSystem(
`Product skill directory (Workspace-owned). Full bodies are available only through on-demand loading.\n${formatProductSkillDirectory(snapshot)}`,
);
args.messageList.addSystem(
"To discover an applicable product procedure, call search_skills. Load only the procedure needed for the current task with load_skill.",
);
for (const [skillName, skill] of state.loaded) {
args.messageList.addSystem(
`[Skill: ${skillName}; revision: ${skill.revision}]\n\n${skill.instructions}`,
);
}

const searchSkills = createTool({
id: "search_skills",
description:
"Search reviewed product research skills applicable to the current runtime capabilities.",
inputSchema: z.object({ query: z.string() }),
outputSchema: z.object({
results: z.array(
z.object({
name: z.string(),
description: z.string(),
revision: z.string(),
}),
),
status: z.enum(["complete", "incomplete"]),
message: z.string(),
}),
execute: async ({ query }) => {
const normalizedQuery = query.trim();
if (!normalizedQuery) {
return {
results: [],
status: "complete" as const,
message: "A non-empty skill search query is required.",
};
}
const current = await this.registry.list({
query: normalizedQuery,
limit: 5,
capabilities: capabilitiesForRequest(),
});
return {
results: current.skills.map((skill) => ({
name: skill.id,
description: skill.description.slice(0, 150),
revision: skill.revision,
})),
status: current.status,
message:
current.status === "incomplete"
? `Skill discovery is incomplete: ${current.diagnostics.join("; ")}`
: `Found ${current.skills.length} applicable skill(s).`,
};
},
});

const loadSkill = createTool({
id: "load_skill",
description:
"Load one reviewed product skill after revalidating current visibility and capabilities.",
inputSchema: z.object({ skillName: z.string() }),
outputSchema: z.object({
success: z.boolean(),
status: z.enum(["complete", "incomplete"]),
message: z.string(),
skillName: z.string().optional(),
revision: z.string().optional(),
}),
execute: async ({ skillName }) => {
const normalizedName = skillName.trim();
const current = await this.registry.list({
query: normalizedName,
includeDetails: true,
limit: 100,
capabilities: capabilitiesForRequest(),
});
const skill = current.skills.find(
(entry) => entry.id === normalizedName,
);
if (current.status !== "complete" || !skill) {
state.loaded.delete(normalizedName);
return {
success: false,
status: current.status,
message:
current.status === "incomplete"
? `Skill discovery is incomplete: ${current.diagnostics.join("; ")}`
: `Skill "${normalizedName}" is unavailable or not applicable to the current runtime capabilities.`,
};
}

state.loaded.set(skill.id, {
revision: skill.revision,
instructions: skill.detail,
});
return {
success: true,
status: "complete" as const,
message: `Skill "${skill.id}" loaded for the current snapshot.`,
skillName: skill.id,
revision: skill.revision,
};
},
});

return {
tools: {
...(args.tools ?? {}),
search_skills: searchSkills,
load_skill: loadSkill,
},
};
}
}

const nativeSecurityResearchSkillSearchProcessor = new SkillSearchProcessor({
workspace: securityResearchWorkspace,
search: {
topK: 2,
minScore: 0,
},
ttl: 60 * 60 * 1000,
});

export const securityResearchSkillSearchProcessor = new RuntimeToolProfileSkillSearchProcessor(
nativeSecurityResearchSkillSearchProcessor,
function readRuntimeSkillCapabilities(value: unknown) {
if (!Array.isArray(value)) return [];
return [
...new Set(
value.filter((item): item is string => typeof item === "string"),
),
].sort();
}

function capabilityKey(capabilities: readonly string[]) {
return capabilities.join(",");
}

function readThreadId(args: ProcessInputStepArgs) {
for (const key of ["mastra__threadId", "threadId", "workspaceThreadId"]) {
const value = readRequestContext(args.requestContext, key);
if (typeof value === "string" && value.trim()) return value;
}
return "default";
}

const workspaceSkills = securityResearchWorkspace.skills;
if (!workspaceSkills) {
throw new Error("Security research Workspace must configure product skills.");
}
const productSkillRegistry = createProductSkillRegistry(
workspaceSkills,
SECURITY_RESEARCH_SKILLS_ROOT,
);

export const securityResearchSkillInputProcessors = [securityResearchSkillSearchProcessor];
export const securityResearchSkillSearchProcessor =
new RegistryBackedSkillSearchProcessor(productSkillRegistry);

export const securityResearchSkillInputProcessors = [
securityResearchSkillSearchProcessor,
];
Loading