Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
57 changes: 54 additions & 3 deletions packages/apollo-react/src/canvas/components/Guardrails/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,8 @@ read-only governance guardrails a policy enforces), plus the leaves the sections
`GuardrailStatusChip`, `GuardrailStatusBanner`, `GuardrailFileSupportIndicator` and
`MixedScopesBanner`.
`GuardrailScopeSelector`, the builder's scope and tool targeting field, is exported on its own
too.
too. The classifier guardrail adds `useClassifierQuestions`, two editors and
`useClassifierGuardrailParameters`.

## Hover and focus, family-wide

Expand Down Expand Up @@ -140,17 +141,23 @@ moved is the six validators both products had already transcribed by hand, where
copies in sync is what produced `finNationalId` in one product and `fiNationalId` in the
other. The components are unchanged: they still resolve nothing and render what they are
handed, so a host that would rather keep its own table simply does not call
`enrichGuardrailDefinitions`. A validator only one product ships (sentiment, Agents only so
far) takes that product's wording verbatim, so its own screens and these read the same.
`enrichGuardrailDefinitions`. A validator only one product ships (sentiment and the
classifier, Agents only so far) takes that product's wording verbatim, so its own screens and
these read the same.

Message ids use the raw wire values, never a transcribed slug:

```
guardrails.definitions.<validator>.display-name | .description | .usage-note
guardrails.definitions.<validator>.param.<paramId>.label | .tooltip
guardrails.definitions.<validator>.option.<paramId>.<RawWireValue>
guardrails.definitions.<validator>.category.<RawGroupId>
```

The last form is `categoryLabels`: headings for the groups a validator's own catalog arrives in,
which no parameter carries, so enrichment leaves them alone. Only the classifier has one, for
the categories its question catalog is grouped in (see [Classifier guardrail](#classifier-guardrail)).

Transcribing is exactly how the two products ended up keying the same Finland entity as
`finNationalId` and `fiNationalId`; `USSocialSecurityNumber` is the value we persist, so it is
also the id.
Expand Down Expand Up @@ -656,6 +663,10 @@ Contract highlights:

- **Owns form state.** Initialized from `definition`/`guardrail`/`defaultName` at mount —
remount with a new `key` to reset (all known hosts already remount per session).
- **Saves a host-rendered threshold map as the host left it.** Every `map-enum` is pruned to
its `keySource` selection; one the builder renders also gets its missing keys filled, so it
matches `MapEnumField`'s rows. A map in `overrideParameterIds` is not filled: the classifier
leaves untuned questions unset, and filling them with `min` would save 0.
- **Owns validation and gates its own Save.** Messages localize through lingui
(overridable per string via `labels`); a host that validates externally (e.g. zod) passes
`errors` — host messages display immediately, win per field, and gate Save. The pure
Expand Down Expand Up @@ -687,6 +698,46 @@ that had it when it opened, looking through open shadow roots (Agents' host is o
returns focus only to a `DialogTrigger`, which no host uses, so without this focus was lost in
light DOM too.

## Classifier guardrail

The `classifier` validator (Agents#6449) asks predefined questions, picked across categories,
and the author's own. Its definition uses ordinary `$parameterType`s, which the generic form
renders as a raw list of question names, so the family ships its editors. The wire, unchanged:

| parameter | holds |
| --- | --- |
| `selectedQuestions` (`enum-list`) | predefined question names |
| `questionThresholds` (`map-enum`) | 0..1 per tuned question; absent runs at the backend's 0.5. A score question stores `level / (levels - 1)` |
| `questions` (`text-list`) | the author's own yes/no questions |
| `questionCriteriaTrue.{i}` / `questionCriteriaFalse.{i}` / `questionThreshold.{i}` | sidecars of `questions[i]`, rewritten on every edit |
| `model` (`enum`, no options) + `modelVendor` (`text`, required) | the host's model picker writes both |

`useClassifierQuestions` loads `GET /api/execution/guardrails/classifier-questions` with the
same context, host shapes and request behaviour as `useGuardrailDefinitions`; the question text
stays the backend's, and only the category headings are curated copy (`categoryLabels`).
`ClassifierQuestionPicker` and `ClassifierCustomQuestionsEditor` are the two editors, and
`readClassifierCustomQuestions` / `writeClassifierCustomQuestions` / `hasClassifierQuestions`
the pure helpers. `useClassifierGuardrailParameters` is the seam both hosts compose:

```tsx
const { categories } = useClassifierQuestions(ctx, { enabled: isClassifier });
const classifier = useClassifierGuardrailParameters({ definition, categories });

<GuardrailBuilder
renderParameter={(c) => classifier.renderParameter(c) ?? renderModelPicker(c)}
overrideParameterIds={[...classifier.overrideParameterIds, 'model']}
onSave={(value) => classifier.validateParameters(value.validatorParameters) && persist(value)}
/>;
```

It renders the picker for `selectedQuestions` (writing both lists at once), the editor for
`questions`, and a message under `modelVendor` when a model was saved without one; `model` falls
through to the host. `validateParameters` is the rule the builder cannot know: at least one
question across both lists. Declare the ids as above, which also keeps an untuned threshold unset
(see GuardrailBuilder). The editors mount through `renderParameter`, the registered
`guardrail-render-parameter` component, not a renderer beside `MetadataForm`; they are not built
into the builder because the catalog and the model picker are the host's.

## CustomGuardrailBuilder

The complete Add/Edit screen for a custom guardrail, `GuardrailBuilder`'s sibling for guardrails
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,138 @@
import type { ClassifierQuestionGroupWire } from '../definitions-wire';

/** Ten of the 26 predefined questions, verbatim from Agents' `ClassifierQuestionPacks`, one or two per category. */
export const CLASSIFIER_QUESTIONS_WIRE: ClassifierQuestionGroupWire[] = [
{
id: 'pii',
questions: [
{
name: 'pii_contact_information',
question:
'Does the content contain personal contact information belonging to an identifiable person, such as an email address, a phone number, or a home address?',
type: 'noul',
criteriaTrue:
'The content includes at least one personal email address, phone number or physical address tied to a person.',
criteriaFalse:
'The content contains no personal contact details, or only clearly fictional/placeholder ones (e.g. user@example.com).',
},
{
name: 'pii_health_information',
question:
'Does the content contain health information about an identifiable person, such as a diagnosis, treatment, medication, or medical record reference?',
type: 'noul',
criteriaTrue:
'The content ties a medical condition, treatment or health record to an identifiable person.',
criteriaFalse:
'The content contains no personal health information; general medical facts not tied to a person count as no.',
},
],
},
{
id: 'harmful_content',
questions: [
{
name: 'harmful_hate',
question:
'Rate the severity of hateful or discriminatory content directed at a person or group based on protected characteristics.',
type: 'score',
levels: [
'No hateful or discriminatory content.',
'Mentions a protected group neutrally or factually, with no negative characterization.',
'Negative stereotyping or demeaning generalizations about a protected group.',
'Direct insults, slurs or dehumanizing language aimed at a person or group.',
'Incitement to discriminate against, harass or harm a person or group.',
],
},
{
name: 'harmful_self_harm',
question: 'Rate the severity of self-harm or suicide content.',
type: 'score',
levels: [
'No self-harm content.',
'Neutral clinical, educational or prevention-oriented discussion of self-harm.',
'First-person expressions of self-harm ideation, without method or encouragement.',
'Encouragement, glorification or instructions for self-harm or suicide.',
],
},
],
},
{
id: 'prompt_injection',
questions: [
{
name: 'injection_jailbreak',
question:
'Does the content attempt to bypass safety or policy restrictions, for example through roleplay framing, hypothetical scenarios, or encoding tricks?',
type: 'noul',
criteriaTrue:
'The content tries to trick the assistant into ignoring safety or policy restrictions.',
criteriaFalse: 'The content contains no attempt to bypass restrictions.',
},
],
},
{
id: 'intellectual_property',
questions: [
{
name: 'ip_verbatim_text',
question:
'Does the content reproduce substantial verbatim passages of copyrighted text, such as song lyrics, book excerpts, or news articles?',
type: 'noul',
criteriaTrue: 'The content contains substantial verbatim copyrighted text.',
criteriaFalse:
'The content contains no substantial verbatim copyrighted passages; short quotes with attribution count as no.',
},
{
name: 'ip_proprietary_code',
question:
'Does the content reproduce source code that appears to be proprietary or under a restrictive license?',
type: 'noul',
criteriaTrue:
'The content contains source code that is plausibly proprietary or restrictively licensed.',
criteriaFalse:
'The content contains no proprietary source code; trivial or generic snippets count as no.',
},
],
},
{
id: 'secrets_and_credentials',
questions: [
{
name: 'secrets_passwords',
question: 'Does the content contain a password or passphrase in plain text?',
type: 'noul',
criteriaTrue: 'The content includes an actual password or passphrase value.',
criteriaFalse:
'The content contains no password values; talking about passwords, reset flows or policies without a value counts as no.',
},
],
},
{
id: 'compliance_commitments',
questions: [
{
name: 'compliance_binding_commitments',
question:
'Does the content promise or commit to something on behalf of the business, such as a refund, discount, guarantee, approval, or deadline?',
type: 'noul',
criteriaTrue:
'The content makes an unconditional promise or commitment the business would have to honor.',
criteriaFalse:
'The content makes no commitment; conditional statements and references to existing policy counts as no.',
},
],
},
{
id: 'scope_and_conduct',
questions: [
{
name: 'conduct_profanity_abuse',
question: 'Does the content contain profanity, insults, or abusive language?',
type: 'noul',
criteriaTrue: 'The content includes profanity or language that insults or demeans someone.',
criteriaFalse:
'The content contains no abusive language; mild colloquial expressions count as no.',
},
],
},
];
Original file line number Diff line number Diff line change
Expand Up @@ -388,6 +388,88 @@ export const SENTIMENT_WIRE: GuardrailDefinitionWire = {
],
};

/** Every predefined classifier question name, `ClassifierQuestionPacks.AllQuestionNames`. */
export const CLASSIFIER_QUESTION_NAMES: string[] = [
'pii_contact_information',
'pii_government_identifiers',
'pii_financial_identifiers',
'pii_health_information',
'harmful_hate',
'harmful_self_harm',
'harmful_sexual',
'harmful_violence',
'harmful_illegal_activity',
'injection_instruction_override',
'injection_jailbreak',
'injection_exfiltration',
'injection_indirect',
'ip_verbatim_text',
'ip_proprietary_code',
'secrets_api_keys_tokens',
'secrets_passwords',
'secrets_private_keys',
'secrets_connection_strings',
'compliance_financial_advice',
'compliance_legal_medical_advice',
'compliance_binding_commitments',
'conduct_off_topic',
'conduct_competitor_steering',
'conduct_politics_religion',
'conduct_profanity_abuse',
];

/** Agents' `OutOfTheBoxGuardrailDefinitions.Classifier` (UiPath/Agents#6449). */
export const CLASSIFIER_WIRE: GuardrailDefinitionWire = {
validator: 'classifier',
allowedScopes: ['Agent', 'Llm', 'Tool'],
status: 'Available',
parameters: [
{
id: 'selectedQuestions',
type: 'enum-list',
required: false,
defaultValue: [],
options: CLASSIFIER_QUESTION_NAMES,
displayName: 'Predefined questions',
description:
'Curated yes/no and severity questions the classifier evaluates against the payload. Pick any across categories, and/or add your own questions below.',
},
{
id: 'questions',
type: 'text-list',
required: false,
defaultValue: [],
maxItems: 10,
maxLength: 500,
},
{
id: 'questionThresholds',
type: 'map-enum',
required: false,
defaultValue: {},
keySource: 'selectedQuestions',
step: 0.05,
min: 0,
max: 1,
},
{
id: 'model',
type: 'enum',
required: true,
defaultValue: '',
options: [],
displayName: 'Model',
},
{
id: 'modelVendor',
type: 'text',
required: true,
defaultValue: '',
displayName: 'Model vendor',
},
],
};

/** A bring-your-own guardrail: manifest copy only, no curated table entry applies. */
export const BYO_WIRE: GuardrailDefinitionWire = {
validator: 'pii_detection',
Expand Down Expand Up @@ -439,6 +521,7 @@ export const ALL_BUILT_IN_WIRE: GuardrailDefinitionWire[] = [
INTELLECTUAL_PROPERTY_WIRE,
LLM_AS_JUDGE_WIRE,
SENTIMENT_WIRE,
CLASSIFIER_WIRE,
];

/**
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,8 @@ import type { GuardrailCopyTable } from '../definitions-copy';
* `frontend-sw/src/components/definition/AddGuardrailPalette/AddGuardrailPalette.utils.tsx`
* (`OOB_GUARDRAILS_I8N`; its `name` / `params[].infoTooltip` / `params[].options` map onto
* `displayName` / `paramTooltips` / `optionLabels` here). The `sentiment` entry is read from
* the same file, as UiPath/Agents#6457 added it.
* the same file, as UiPath/Agents#6457 added it, and the `classifier` entry as
* UiPath/Agents#6449 did, its `categoryLabels` from `resolveClassifierCategoryLabel`.
* - Flow `origin/develop`:
* `packages/canvas/src/components/properties-panel/guardrails/ootb-guardrail-definitions.ts`
* (`buildValidatorDisplayInfo`)
Expand Down Expand Up @@ -343,6 +344,31 @@ export const AGENTS_COPY_EN: GuardrailCopyTable = {
},
},
},
classifier: {
displayName: 'Classifier',
description:
"Evaluate questions against the payload with a non-generative classifier. Pick curated questions across categories, add your own, and flag when any answer reaches that question's threshold.",
paramLabels: {
selectedQuestions: 'Predefined questions',
questions: 'Your questions',
model: 'Model',
},
paramTooltips: {
selectedQuestions:
'Curated questions grouped by category. Pick any across categories; score questions also take a severity cutoff. Add your own questions below.',
questions:
"Yes/no questions about the payload, where 'yes' means the payload should be flagged. Added on top of any predefined questions you selected. Optionally define what counts as yes and as no — the classifier grades against those definitions, which helps on judgment-style questions.",
},
categoryLabels: {
pii: 'PII',
harmful_content: 'Harmful content',
prompt_injection: 'Prompt injection',
intellectual_property: 'Intellectual property',
secrets_and_credentials: 'Secrets & credentials',
compliance_commitments: 'Compliance & commitments',
scope_and_conduct: 'Scope & conduct',
},
},
};

/** Flow `packages/canvas`, `buildValidatorDisplayInfo()`. */
Expand Down
Loading
Loading