Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
20 changes: 15 additions & 5 deletions packages/apollo-react/src/canvas/components/Guardrails/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -581,6 +581,15 @@ Contract highlights:
`formatScope` / `formatAction` to override. Every one of those strings already existed in
the canvas catalog, so defaulting removes a prop an adapter can forget for a visible
regression (`Llm` instead of "LLM calls").
- **The row's summary line reads as the hosts' own guardrail rows do**:
`Scope: LLM calls | Detection: Files | Execution stage: Pre-execution`, with the bars
`aria-hidden`. Detection is what the guardrail reads (`getCentralizedGuardrailDetection`):
the policy's `detectionScope`, or Text when it stored none, as the gateway reads it; `Both`
reads "Text, Files". It shows only on a built-in whose definition still offers the
parameter. Hosts strip it from their definitions where file support is off, and Agents then
reads no file for any guardrail, so the row leaves Detection out rather than claim it. A BYO
row, a validator without the parameter, and a row whose definitions are still loading show
none.
- **A broken BYO configuration gets a chip and a sentence.** The chip is `GuardrailList`'s
own (`Unavailable` / `Disabled`, same ids) and makes the row findable in a long policy; the
sentence under it, which both products already show, says what to do about it.
Expand Down Expand Up @@ -900,11 +909,12 @@ Where it renders depends on `fileSupport` and on the definition's `detectionScop
| `supported: true` | absent | above the parameters, always |
| `supported: false` | any | above the parameters, always |

What Detection scope selects is `getGuardrailDetectionScope(definitions, parameters)`, which reads
it the way the backend does: the stored value, trimmed and case-insensitive, when non-empty,
otherwise the definition's `defaultValue` (Text for the five GA validators, Both for
`llm_as_judge`). Any other value, Flow's legacy `Prompts` included, is Text. The builder
recomputes it on every edit, so the lines follow the select as it changes.
What Detection scope selects is `getGuardrailDetectionScope(definitions, parameters)`: the
stored value, trimmed and case-insensitive, when non-empty, otherwise the definition's
`defaultValue`, which is Text for every validator. Any other value, Flow's legacy `Prompts`
included, is Text. The backend reads a stored value the same way, and an absent one as Text
whatever the definition says (UiPath/Agents#6740). The builder recomputes it on every edit, so
the lines follow the select as it changes.

## GuardrailActionSection

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,7 @@ import {
formatCentralizedExecutionStage,
formatCentralizedScope,
getApplicableCentralizedGuardrails,
getCentralizedGuardrailDetection,
getCentralizedGuardrailDisplay,
getCentralizedGuardrailItemId,
isCentralizedGuardrailConfigMissing,
Expand Down Expand Up @@ -205,6 +206,60 @@ describe('findCentralizedBuiltInDefinition', () => {
});
});

describe('getCentralizedGuardrailDetection', () => {
const offered = [{ validator: 'pii_detection', parameters: [{ id: 'detectionScope' }] }];
const scoped = (value: unknown) =>
guardrail({ parameters: [{ id: 'detectionScope', parameterType: 'enum', value }] });

it.each([
['Files', 'Files'],
['Both', 'Both'],
['Text', 'Text'],
[' files ', 'Files'],
['BOTH', 'Both'],
['Prompts', 'Text'],
['', 'Text'],
[42, 'Text'],
])('reads a stored %j as %s, the way the gateway reads the policy', (value, expected) => {
expect(getCentralizedGuardrailDetection(scoped(value), offered)).toBe(expected);
});

it('reads a built-in saved without a detection scope as Text', () => {
expect(getCentralizedGuardrailDetection(guardrail(), offered)).toBe('Text');
expect(getCentralizedGuardrailDetection(guardrail({ parameters: null }), offered)).toBe('Text');
});

it('has nothing to say while the definitions load or once file support stripped the parameter', () => {
expect(getCentralizedGuardrailDetection(scoped('Files'), undefined)).toBeUndefined();
expect(
getCentralizedGuardrailDetection(scoped('Files'), [{ validator: 'pii_detection' }])
).toBeUndefined();
expect(
getCentralizedGuardrailDetection(scoped('Files'), [
{ validator: 'pii_detection', parameters: [{ id: 'entities' }] },
])
).toBeUndefined();
});

it('matches only the built-in definition of the same validator', () => {
expect(
getCentralizedGuardrailDetection(scoped('Files'), [
{ validator: 'harmful_content', parameters: [{ id: 'detectionScope' }] },
{
validator: 'pii_detection',
byoValidatorName: 'Acme PII',
parameters: [{ id: 'detectionScope' }],
},
])
).toBeUndefined();
});

it('never reports one for a BYO guardrail', () => {
const byo = { ...scoped('Files'), isByo: true, name: 'Acme PII' };
expect(getCentralizedGuardrailDetection(byo, offered)).toBeUndefined();
});
});

describe('isCentralizedGuardrailConfigMissing', () => {
const identity = { validator: 'pii_detection', name: 'Acme PII', isByo: true };

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -59,18 +59,51 @@ export function findCentralizedByoDefinition<T extends CentralizedGuardrailDefin
);
}

/** A built-in's definition, used only to label its configuration rows. */
export function findCentralizedBuiltInDefinition<T extends CentralizedGuardrailDefinition>(
guardrail: CentralizedGuardrailIdentity,
definitions: readonly T[] | undefined
): T | undefined {
/** A built-in's definition, to label its configuration rows and gate its detection scope. */
export function findCentralizedBuiltInDefinition<
T extends Pick<CentralizedGuardrailDefinition, 'validator' | 'byoValidatorName'>,
>(guardrail: CentralizedGuardrailIdentity, definitions: readonly T[] | undefined): T | undefined {
if (guardrail.isByo) return undefined;
return definitions?.find(
(definition) =>
definition.validator === guardrail.validator && definition.byoValidatorName === undefined
);
}

/** The definition fields `getCentralizedGuardrailDetection` reads, so any host's shape fits. */
export type CentralizedDetectionDefinition = Pick<
CentralizedGuardrailDefinition,
'validator' | 'byoValidatorName'
> & { parameters?: readonly { id: string }[] };

/**
* What a built-in reads, for its row. Agents stamps the policy's detection scope on every
* centralized built-in, so it shows only while the built-in's definition still offers the
* parameter: hosts strip it where file support is off, and then no file is read. Absent, blank
* or unknown is Text, as the gateway reads the policy.
*/
export function getCentralizedGuardrailDetection(
guardrail: CentralizedGuardrailIdentity & Pick<CentralizedGuardrail, 'parameters'>,
definitions: readonly CentralizedDetectionDefinition[] | undefined
): GuardrailDetectionScope | undefined {
const definition = findCentralizedBuiltInDefinition(guardrail, definitions);
const offered = definition?.parameters?.some(
(parameter) => parameter.id === GUARDRAIL_DETECTION_SCOPE_PARAMETER_ID
);
if (!offered) return undefined;
const value = guardrail.parameters?.find(
(parameter) => parameter.id === GUARDRAIL_DETECTION_SCOPE_PARAMETER_ID
)?.value;
switch (typeof value === 'string' ? value.trim().toLowerCase() : '') {
case 'files':
return 'Files';
case 'both':
return 'Both';
default:
return 'Text';
}
}

/** Whether a BYO configuration is gone. `undefined` definitions (still loading) report `false`. */
export function isCentralizedGuardrailConfigMissing(
guardrail: CentralizedGuardrailIdentity,
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -26,6 +26,18 @@ const BYO_DEFINITION: CentralizedGuardrailDefinition = {
description: 'Acme runs its own detector.',
};

const withDetectionScope = (validator: string): CentralizedGuardrailDefinition => ({
validator,
parameters: [{ id: 'detectionScope', type: 'enum' }],
});

const detectionScope = (value: string): Partial<CentralizedGuardrail> => ({
parameters: [{ id: 'detectionScope', parameterType: 'enum', value }],
});

const summaries = (container: HTMLElement) =>
Array.from(container.querySelectorAll('[data-slot="centralized-guardrail-summary"]'));

describe('CentralizedGuardrailsSection', () => {
it('renders nothing at all when the policy enforces none, as both products do', () => {
const { container } = render(
Expand Down Expand Up @@ -89,26 +101,85 @@ describe('CentralizedGuardrailsSection', () => {
);
});

it('localizes the scopes and the execution stage without a formatScope prop', () => {
render(<CentralizedGuardrailsSection guardrails={[guardrail()]} policyName="Acme policy" />);
it('localizes the scope and the execution stage without a formatScope prop', () => {
const { container } = render(
<CentralizedGuardrailsSection guardrails={[guardrail()]} policyName="Acme policy" />
);

expect(
screen.getByText('Scopes: Agent, LLM calls · Execution stage: Pre-execution')
).toBeInTheDocument();
const [summary] = summaries(container);
expect(summary).toHaveTextContent('Scope: Agent, LLM calls | Execution stage: Pre-execution');
// The separator is decoration; the parts are named by their own labels.
expect(summary?.querySelector('[aria-hidden="true"]')).toHaveTextContent('|');
});

it('lets a host replace the scope names', () => {
render(
const { container } = render(
<CentralizedGuardrailsSection
guardrails={[guardrail()]}
policyName="Acme policy"
formatScope={(scope) => scope.toUpperCase()}
/>
);

expect(
screen.getByText('Scopes: AGENT, LLM · Execution stage: Pre-execution')
).toBeInTheDocument();
expect(summaries(container)[0]).toHaveTextContent(
'Scope: AGENT, LLM | Execution stage: Pre-execution'
);
});

it('says what a built-in reads while its definition offers the detection scope', () => {
const { container } = render(
<CentralizedGuardrailsSection
guardrails={[
guardrail({ scopes: ['Llm'], ...detectionScope('Files') }),
guardrail({ validator: 'llm_as_judge', scopes: ['Llm'], ...detectionScope('Both') }),
guardrail({ validator: 'harmful_content', scopes: ['Llm'] }),
]}
definitions={[
withDetectionScope('pii_detection'),
withDetectionScope('llm_as_judge'),
withDetectionScope('harmful_content'),
]}
policyName="Acme policy"
/>
);

expect(summaries(container).map((summary) => summary.textContent)).toEqual([
'Scope: LLM calls | Detection: Files | Execution stage: Pre-execution',
'Scope: LLM calls | Detection: Text, Files | Execution stage: Pre-execution',
// Saved without a scope: the gateway reads it as text.
'Scope: LLM calls | Detection: Text | Execution stage: Pre-execution',
]);
});

it('leaves the detection scope out where the definitions do not offer it', () => {
// Stripped by the host where file support is off, still loading, a BYO row, and a
// validator with no such parameter.
const files = detectionScope('Files');
const { container, rerender } = render(
<CentralizedGuardrailsSection
guardrails={[guardrail(files)]}
definitions={[{ validator: 'pii_detection', parameters: [] }]}
policyName="Acme policy"
/>
);
expect(summaries(container)[0]).not.toHaveTextContent('Detection');

rerender(
<CentralizedGuardrailsSection guardrails={[guardrail(files)]} policyName="Acme policy" />
);
expect(summaries(container)[0]).not.toHaveTextContent('Detection');

rerender(
<CentralizedGuardrailsSection
guardrails={[byoGuardrail(files), guardrail({ validator: 'sentiment', ...files })]}
definitions={[
{ ...BYO_DEFINITION, parameters: [{ id: 'detectionScope', type: 'enum' }] },
withDetectionScope('pii_detection'),
]}
policyName="Acme policy"
/>
);
for (const summary of summaries(container)) expect(summary).not.toHaveTextContent('Detection');
});

it('says nothing about a missing configuration while the catalog is still loading', () => {
Expand Down Expand Up @@ -245,9 +316,10 @@ describe('CentralizedGuardrailsSection', () => {
</ApI18nProvider>
);

// The row's metadata line, through the builder ids this block reuses. The section's own
// ids render English until `chore(l10n): sync from Localization` reaches them.
expect(screen.getByText(/スコープ/)).toBeInTheDocument();
// The row's summary line: the reused builder scope names and the section's translated stage.
// Newer ids render English until `chore(l10n): sync from Localization` reaches them.
expect(screen.getByText(/エージェント, LLM の呼び出し/)).toBeInTheDocument();
expect(screen.getByText(/実行ステージ: 実行前/)).toBeInTheDocument();
});

it('has no axe violations', async () => {
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,7 @@ import type { GuardrailScope } from './builder-types';
import {
findCentralizedByoDefinition,
formatCentralizedScope,
getCentralizedGuardrailDetection,
getCentralizedGuardrailDisplay,
getCentralizedGuardrailItemId,
isCentralizedGuardrailConfigMissing,
Expand All @@ -23,7 +24,10 @@ export interface CentralizedGuardrailsSectionProps<
guardrails: CentralizedGuardrail[];
/** Name of the AI Trust Layer policy enforcing them; shown in the caption. */
policyName: string;
/** For provider names and configuration status. `undefined` while the catalog loads. */
/**
* For provider names, configuration status and whether a row shows its detection scope: pass
* them after file-support stripping. `undefined` while the catalog loads.
*/
definitions?: TDefinition[];
/** The host opens the details (a dialog, a panel overlay). */
onSelect?: (guardrail: CentralizedGuardrail) => void;
Expand Down Expand Up @@ -131,6 +135,7 @@ export function CentralizedGuardrailsSection<
scopeNames={guardrail.scopes.map(
(scope) => formatScope?.(scope) ?? formatCentralizedScope(scope, labels)
)}
detection={getCentralizedGuardrailDetection(guardrail, definitions)}
labels={labels}
onSelect={onSelect === undefined ? undefined : () => onSelect(guardrail)}
/>
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -281,6 +281,51 @@ export const StillLoading: Story = {
},
};

const detectionScopeParameter = { id: 'detectionScope', type: 'enum', label: 'Detection scope' };

/** The catalog as a host passes it where file support is on. */
const definitionsWithFileSupport: CentralizedGuardrailDefinition[] = [
...definitions.map((definition) =>
definition.byoValidatorName === undefined &&
(definition.validator === 'pii_detection' || definition.validator === 'harmful_content')
? { ...definition, parameters: [...(definition.parameters ?? []), detectionScopeParameter] }
: definition
),
{ validator: 'llm_as_judge', parameters: [detectionScopeParameter] },
];

export const DetectionScope: Story = {
name: 'What each guardrail reads',
args: {
guardrails: [piiFilesGuardrail, judgeGuardrail, harmfulContentGuardrail, byoGuardrail],
definitions: definitionsWithFileSupport,
policyName: 'Acme production policy',
docsHref: DOCS_HREF,
onSelect: noop,
},
parameters: {
docs: {
description: {
story:
'Where file support is on, the definitions a host passes still offer the detection scope, so each built-in row says what it reads: the policy value, or Text when the policy stored none. A BYO row never shows one.',
},
},
},
};

export const DetectionScopeWithoutFileSupport: Story = {
name: 'What each guardrail reads, file support off',
args: { ...DetectionScope.args, definitions },
parameters: {
docs: {
description: {
story:
'Where file support is off, the host strips the detection scope from its definitions and no guardrail reads files, so the rows leave it out even though the policy still carries a value.',
},
},
},
};

export const InsideHostChrome: Story = {
name: 'Inside a host section',
args: {
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -78,6 +78,11 @@ const HOST_COPY: Partial<Record<keyof CentralizedGuardrailsLabels, HostCopy>> =
detectionScopeText: { agents: 'Text only' },
detectionScopeFiles: { agents: 'Files only' },
detectionScopeBoth: { agents: 'Text and files' },
// Both products' own guardrail rows (Agents#6782, Flow#5421), whose wording the row adopts.
scope: { agents: 'Scope', flow: 'Scope' },
detection: { agents: 'Detection', flow: 'Detection' },
detectionText: { agents: 'Text', flow: 'Text' },
detectionFiles: { agents: 'Files', flow: 'Files' },
};

interface CopyDivergence {
Expand Down
Loading
Loading