diff --git a/skills/uipath-human-in-the-loop/SKILL.md b/skills/uipath-human-in-the-loop/SKILL.md index 87bf708c33..fc8f315128 100644 --- a/skills/uipath-human-in-the-loop/SKILL.md +++ b/skills/uipath-human-in-the-loop/SKILL.md @@ -394,7 +394,7 @@ Notes on what's easy to get wrong: - **`` (last child of ``) and `` (sibling of ``, not inside it) are both required.** Without them the "Edit Schema" canvas in Studio Web doesn't open at all — confirmed by direct reproduction. This mirrors the Case surface's `data.inputSchema`/`data.inputs[]` requirement — see [hitl-casetask-action.md § Step 3](references/hitl-casetask-action.md) for the parallel Case shape and full field-by-field rationale. - **The `Action` output requires a matching process-level variable declaration** — add `` inside the process's top-level `` block. - **A separate `.hitl.json` sidecar file is required**, same shape as the Case surface's (`title`, `fields[]` with `direction`/optional `colSpan`/`binding` or `variable`, `outcomes[]` with **no** `action` key, `schemaId`) — see [hitl-casetask-action.md § Step 2](references/hitl-casetask-action.md) for the exact shape; it's identical across both surfaces. -- **`_schemaFileId` cannot be authored blind — it is a server-assigned foreign key, not a UUID you invent.** A placeholder value makes "Edit Schema" fail silently (a `404` on Studio Web's internal `FileOperations/File/Rename` call, confirmed by direct reproduction). There is no `uip` CLI command that resolves this today. **Never block on this.** Write a fresh placeholder UUID v4, finish the rest of the node, validate, and move on — do not attempt the live reconciliation yourself and do not stop to ask about it. State plainly in the final report: "This task's schema was written with a placeholder `_schemaFileId`, so Studio Web's 'Edit Schema' canvas won't open for it yet. Making it editable requires resolving the real file ID Studio Web assigns to the `.hitl.json` after upload (`GET /api/Project/{projectId}/FileOperations/Structure`) and pushing it back with a targeted single-file update (`PUT /api/Project/{projectId}/FileOperations/File/{fileId}`) — not another whole-project `uip solution upload`, which mints a new random ID for every file and undoes the fix. Ask me to do this reconciliation if you want the schema editable in Studio Web." This is a real product/tooling gap the user can act on later, not something to resolve mid-task. +- **`_schemaFileId` cannot be authored blind — it is a server-assigned foreign key, not a UUID you invent.** A placeholder value makes "Edit Schema" fail silently (a `404` on Studio Web's internal `FileOperations/File/Rename` call, confirmed by direct reproduction). There is no `uip` CLI command that resolves this today. **Never block on this.** Write a fresh placeholder UUID v4, finish the rest of the node, validate, and move on — do not attempt the live reconciliation yourself and do not stop to ask about it. State plainly in the final report: "This task's schema was written with a placeholder `_schemaFileId`, so Studio Web's 'Edit Schema' canvas won't open for it yet. Making it editable requires resolving the real file ID Studio Web assigns to the `.hitl.json` after upload (`GET /api/Project/{projectId}/FileOperations/Structure`) and pushing it back with a targeted single-file update (`PUT /api/Project/{projectId}/FileOperations/File/{fileId}`) — not another whole-project `uip solution upload`, which re-pushes every file and mints a fresh random file ID for each one, undoing the fix. Ask me to do this reconciliation if you want the schema editable in Studio Web." This is a real product/tooling gap the user can act on later, not something to resolve mid-task. - **Diagram-interchange edges are required for the connector lines to render, even though the logical `bpmn:sequenceFlow`s already exist.** Every `bpmn:sequenceFlow` needs a matching `bpmndi:BPMNEdge` (with two `di:waypoint` points, aligned to the source/target shapes' connecting edges) in the `bpmndi:BPMNPlane` — omitting it leaves the nodes logically connected but visually disconnected in the canvas. Confirmed by direct reproduction. Design the schema per Step 4b — never block waiting on the user, per Critical Rule 1 — then validate frequently (`uip maestro bpmn validate .bpmn --output json`) while wiring the node so any shape mistakes surface immediately rather than at deploy time. In Maestro, field names in `outputs`/`inOuts` must exactly match declared process variable names and types. diff --git a/skills/uipath-human-in-the-loop/references/hitl-casetask-action.md b/skills/uipath-human-in-the-loop/references/hitl-casetask-action.md index c6d9b0525a..6dc831253c 100644 --- a/skills/uipath-human-in-the-loop/references/hitl-casetask-action.md +++ b/skills/uipath-human-in-the-loop/references/hitl-casetask-action.md @@ -230,7 +230,7 @@ The file uses a **unified `fields[]` array** — every field has a `direction` p > 1. Upload the project once with any placeholder value in `_schemaFileId` (`uip solution upload`). > 2. Look up the real ID Studio Web assigned to the `.hitl.json` file via `GET /api/Project/{projectId}/FileOperations/Structure` (find the entry whose `name` matches the `.hitl.json` filename — an internal Studio Web REST endpoint, not a `uip` CLI verb). > 3. Patch `_schemaFileId` in `caseplan.json` to that real ID. -> 4. Push the corrected `caseplan.json` back with a **targeted single-file update** — `PUT /api/Project/{projectId}/FileOperations/File/{fileId}` (same file's own real ID) — **not** another whole-project `uip solution upload`. A second whole-project upload re-imports everything and mints a **new** random file ID for every file, including `.hitl.json`, immediately invalidating whatever you just patched. +> 4. Push the corrected `caseplan.json` back with a **targeted single-file update** — `PUT /api/Project/{projectId}/FileOperations/File/{fileId}` (same file's own real ID) — **not** another whole-project `uip solution upload`. A second whole-project upload re-pushes every file and mints a **new** random file ID for each one, including `.hitl.json`, immediately invalidating whatever you just patched. > > This is a real gap, not just a documentation gap: today there is no supported CLI path to make a freshly-authored QuickForm task's schema editable in Studio Web without dropping to this undocumented internal API. It's a follow-up the user can request, never a mid-task blocker. @@ -246,7 +246,7 @@ Read available case variables from the top-level `variables` field in `caseplan. } ``` -For cross-task references, source values come from upstream task `outputs[].var` — see [bindings-and-expressions.md](../../../uipath-maestro-case/references/bindings-and-expressions.md) in the case skill for the full discovery procedure. +For cross-task references, source values come from upstream task `outputs[].var` — see [bindings-and-expressions.md](../../uipath-maestro-case/references/bindings-and-expressions.md) in the case skill for the full discovery procedure. > **No root-level bindings needed for QuickForm.** Unlike App-based (Path 2), QuickForm does **not** add entries to the top-level `bindings[]` array. @@ -272,7 +272,7 @@ Each field in `outputs[]` and `inOuts[]` exposes its value downstream via the fi Downstream task input value: `"=vars.decision"`. The selected outcome is available via the task's `Action` output (see Step 3). -For the full cross-task wiring procedure, see [bindings-and-expressions.md](../../../uipath-maestro-case/references/bindings-and-expressions.md). +For the full cross-task wiring procedure, see [bindings-and-expressions.md](../../uipath-maestro-case/references/bindings-and-expressions.md). --- @@ -330,7 +330,7 @@ Add 2 entries to the top-level `bindings[]` array — one for `name` and one for `resourceKey` = `.`. Binding IDs: `b` + 8 chars. -For the full binding procedure, see [bindings/impl-json.md](../../../uipath-maestro-case/references/plugins/variables/bindings/impl-json.md) in the case skill. +For the full binding procedure, see [bindings/impl-json.md](../../uipath-maestro-case/references/plugins/variables/bindings/impl-json.md) in the case skill. ### Step 4 — Write the Task @@ -369,7 +369,7 @@ For the full binding procedure, see [bindings/impl-json.md](../../../uipath-maes > **`hitlType` is `"custom"` for app-based tasks.** App-based tasks have no schema file at all — the form is defined by the deployed app itself, not by a schema document. **Never write `_schemaFileId` or `hitlSchemaId` for an app-based task** — those two context entries exist only on the QuickForm path (Path 1), where they identify the `.hitl.json` schema file. An app-based task is fully described by `data.name`/`data.folderPath` (the bindings to the deployed app) plus `data.inputs[]`/`data.outputs[]` (from `tasks describe`) — nothing else identifies its "schema." -For the full `inputs[]`/`outputs[]` variable shapes, see [action/impl-json.md](../../../uipath-maestro-case/references/plugins/tasks/action/impl-json.md). +For the full `inputs[]`/`outputs[]` variable shapes, see [action/impl-json.md](../../uipath-maestro-case/references/plugins/tasks/action/impl-json.md). --- @@ -412,4 +412,4 @@ Downstream task input: `"value": "=vars.decision"`. Downstream task input: `"value": "=vars.decisionVar"`. -For the full cross-task wiring procedure, see [bindings-and-expressions.md](../../../uipath-maestro-case/references/bindings-and-expressions.md). +For the full cross-task wiring procedure, see [bindings-and-expressions.md](../../uipath-maestro-case/references/bindings-and-expressions.md). diff --git a/skills/uipath-ixp/SKILL.md b/skills/uipath-ixp/SKILL.md index a7d6c4a0d6..52e78cf346 100644 --- a/skills/uipath-ixp/SKILL.md +++ b/skills/uipath-ixp/SKILL.md @@ -102,7 +102,7 @@ If the user provides a taxonomy file, use `--skip-taxonomy` and `import-taxonomy | "Confirm one line item / extraction" / "Confirm only this occurrence" | `uip ixp labellings confirm --group --occurrence [--fields ] --output json` — targets one specific extraction of a repeatable field group (0-based index **from the latest `get-predictions`**). Without `--fields`, confirms every predicted field in that occurrence; with `--fields`, only those. Other occurrences untouched. Confirming renumbers the group on the next read (the confirmed row moves to `Occurrence` 0) — so batch multiple occurrences into one `--updates '[…]'` call rather than chaining `--occurrence` calls off a single read. See Critical Rules 13 and 18. | | "Unconfirm one line item / extraction" / "Roll back only this occurrence" | `uip ixp labellings unconfirm --group --occurrence [--fields ] --output json` — rolls back one specific extraction of a repeatable field group (0-based index, same as `get-predictions`/`confirm`). Without `--fields`, unconfirms every annotated field in that occurrence; with `--fields`, only those. Other occurrences untouched. **Re-read `get-predictions` first** — on a partly-confirmed group the confirmed rows sort to the front, so the index that confirmed a row is usually not the index that rolls it back (Critical Rule 18). For several occurrences in one call, use `--updates '[…]'` instead. See Critical Rule 14. | | "Set overall extraction instructions" / "Update project prompt" | `uip ixp projects update-prompt --prompt "" --output json` — replaces the taxonomy-wide prompt (the "Overall extraction instructions" field in the IXP UI). Distinct from `fields update-prompts` (per-field) and `groups update-prompts` (per-field-group). | -| "How is this project performing?" / "What's the F1?" | Resolve the live version with `uip ixp projects list-models --output json`, then `uip ixp projects get-metrics --model-version --output json` (Critical Rule 21). If Data is `{ Metrics: null }` the model isn't validated yet — report that and stop. If the call instead returns `Result: Failure` with `ErrorCode: not_found` (a project with no trained model yet, e.g. no confirmed labellings), treat it the same way — report "no metrics yet" and stop. Otherwise Data is flat; report in order: (1) which version the scores belong to + `TrainedTime`; (2) overall `ProjectScore`/`ProjectScoreQuality`; (3) per-group scores from `FieldGroups[]` (F1/Precision/Recall); (4) per-field scores from `Fields[]`, sorted lowest-F1 first — F1 **with** its `Precision`/`Recall` (a low F1 means the opposite fix depending on which side is short), plus `Annotations` (the sample size behind the F1) and `ErrorRate` (= errors/`Annotations` — it counts misses, so it is **not** `1 - Precision`). Ignore the `Quality` labels (derived, inconsistent scales — [Improve Prompts Guide § What get-metrics returns](references/improve-prompts-guide.md#what-get-metrics-returns-and-which-values-decide)). State numbers plainly; no "good enough" judgement unless asked; route low scores to [Improve Prompts Guide](references/improve-prompts-guide.md). Answer from these calls only — no ad-hoc discovery (Critical Rule #1). | +| "How is this project performing?" / "What's the F1?" | Resolve the live version with `uip ixp projects list-models --output json`, then `uip ixp projects get-metrics --model-version --output json` (Critical Rule 21). If Data is `{ Metrics: null }` the model isn't validated yet — report that and stop. If the call instead returns `Result: Failure` with `ErrorCode: not_found` (a project with no trained model yet, e.g. no confirmed labellings), treat it the same way — report "no metrics yet" and stop. Otherwise Data is flat; report in order: (1) which version the scores belong to + `TrainedTime`; (2) overall `ProjectScore`/`ProjectScoreQuality`; (3) per-group scores from `FieldGroups[]` (F1/Precision/Recall); (4) per-field scores from `Fields[]`, sorted lowest-F1 first — F1 **with** its `Precision`/`Recall` (a low F1 means the opposite fix depending on which side is short), plus `Annotations` (the sample size behind the F1) and `ErrorRate` (= errors/`Annotations` — it counts misses, so it is **not** `1 - Precision`). Ignore the `Quality` labels (derived, inconsistent scales — [Improve Prompts Guide § What get-metrics returns](references/improve-prompts-guide.md#what-get-metrics-returns-and-which-values-decide)). Name and compare fields from `Fields[]` alone — [Improve Prompts Guide § 1a](references/improve-prompts-guide.md#1a-get-baseline-metrics). State numbers plainly; no "good enough" judgement unless asked; route low scores to [Improve Prompts Guide](references/improve-prompts-guide.md). Answer from these calls only — no ad-hoc discovery (Critical Rule #1). | | "Describe this project" / "What's in it?" | Three calls, reported in order: (1) identity — `Title`/`Name` from `uip ixp projects get --output json`; (2) current model **version** — live/published + `TrainedTime` from `list-models` (the trained version, not the configured extraction model — for that see the row above); (3) taxonomy — label-group/field counts from `uip ixp projects get-taxonomy --output json` (raw artifact: counts live under `Data.dataset.label_groups` and `Data.dataset.entity_defs`, snake_case). Fold in performance (above) only if asked. Do NOT page `documents list` (its `Data` is a paged `{ Documents, Total, Offset, Limit }` envelope — use `Total` for a count) or read deployment bindings. Answer from these calls only (Critical Rule #1). | ## Common Pitfalls diff --git a/skills/uipath-ixp/references/cli-reference.md b/skills/uipath-ixp/references/cli-reference.md index 038a30911f..64ae815efe 100644 --- a/skills/uipath-ixp/references/cli-reference.md +++ b/skills/uipath-ixp/references/cli-reference.md @@ -15,7 +15,7 @@ All commands use `uip ixp` prefix. Always append `--output json` when parsing ou | `uip ixp projects update-title "" --output json` | Update the display title of a project | | `uip ixp projects update-prompt --prompt "" --output json` | Update the project's **Overall extraction instructions** — the taxonomy-wide prompt the model sees on every extraction (the field at the top of the IXP UI's Manage Taxonomy page). Distinct from per-field-group prompts (`groups update-prompts`) and per-field prompts (`fields update-prompts`). Replaces the existing value. | | `uip ixp projects get-taxonomy --output json` | Export the raw IXP taxonomy artifact. Data is `{ status, dataset: { entity_defs, label_groups } }` — read `entity_defs` and `label_groups` under `dataset`. Intended for re-import (see `import-taxonomy`), not a human-readable view. `dataset` also carries `_model_config`, the only read path for the configured extraction model and pre-processing — see [Reading the current model and pre-processing](#reading-the-current-model-and-pre-processing). | -| `uip ixp projects get-metrics [--model-version ] --output json` | Get validation metrics. **Validated model →** flat Data: `ProjectScore`, `ProjectScoreQuality`, `ValidatedDocuments`, `ModelVersion`, plus per-group `FieldGroups[]` (`FieldGroup`, `F1`, `Precision`, `Recall`, `ErrorRate`, `Documents`) and per-field `Fields[]` (`FieldGroup`, `FieldId`, `Name`, `F1`, `Precision`, `Recall`, `ErrorRate`, `Documents`, `Annotations`, `Quality`). `Name` is the field's display name resolved from the taxonomy — report on it, but compare on `FieldId`, which is the stable key; it is `null` when the service could not resolve it (e.g. the field was deleted after that version was scored). Display names are unique only within a group, so qualify as ` / ` when two fields share one. Scores are surfaced at the backend's own precision — long tails like `0.824999988079071` are its float32 arithmetic widened to double, not extra accuracy; round when you display them, and compare the raw values. **Trained but not yet validated →** Data is `{ Metrics: null }` (not an error). **No trained model yet (e.g. a project with no confirmed labellings) →** the call returns a failure envelope `Result: Failure` with `ErrorCode: not_found` (no `Data`), NOT `{ Metrics: null }` — treat it as "no metrics yet". **Defaults to the LATEST TRAINED version, which is NOT necessarily the published/live one** — resolve the version from `list-models` and pass it as `--model-version ` whenever you report a score, so the numbers and the version identity match (SKILL.md Critical Rule 21). **Any version the backend ever scored is readable**, including older ones `list-models` no longer lists — that is what makes a version-to-version comparison possible; `not_found` on a version means the backend never scored it, not that it aged out. Field semantics — which values decide and which are derived — are in [Improve Prompts Guide § What get-metrics returns](improve-prompts-guide.md#what-get-metrics-returns-and-which-values-decide). `ErrorRate` is `errors / Annotations` (it counts misses — not `1 - Precision`); the `Quality`/`ProjectScoreQuality` labels use inconsistent scales — never gate on them. | +| `uip ixp projects get-metrics [--model-version ] --output json` | Get validation metrics. **Validated model →** flat Data: `ProjectScore`, `ProjectScoreQuality`, `ValidatedDocuments`, `ModelVersion`, plus per-group `FieldGroups[]` (`FieldGroup`, `F1`, `Precision`, `Recall`, `ErrorRate`, `Documents`) and per-field `Fields[]` (`FieldGroup`, `FieldId`, `Name`, `F1`, `Precision`, `Recall`, `ErrorRate`, `Documents`, `Annotations`, `Quality`). `Name` is the field's display name in that version's taxonomy, so no taxonomy read is needed — do NOT fetch one. Report `Name`, but compare on `FieldId`, which is the stable key: two versions can return different names for the same field. `Name` is `null` only when the version's taxonomy is unreadable, and then every name in the response is `null` — use `FieldId` and re-run if you need names. Display names are unique only within a group, so qualify as ` / ` when two fields share one. Scores are surfaced at the backend's own precision — long tails like `0.824999988079071` are its float32 arithmetic widened to double, not extra accuracy; round when you display them, and compare the raw values. **Trained but not yet validated →** Data is `{ Metrics: null }` (not an error). **No trained model yet (e.g. a project with no confirmed labellings) →** the call returns a failure envelope `Result: Failure` with `ErrorCode: not_found` (no `Data`), NOT `{ Metrics: null }` — treat it as "no metrics yet". **Defaults to the LATEST TRAINED version, which is NOT necessarily the published/live one** — resolve the version from `list-models` and pass it as `--model-version ` whenever you report a score, so the numbers and the version identity match (SKILL.md Critical Rule 21). **Any version the backend ever scored is readable**, including older ones `list-models` no longer lists — that is what makes a version-to-version comparison possible; `not_found` on a version means the backend never scored it, not that it aged out. Field semantics — which values decide and which are derived — are in [Improve Prompts Guide § What get-metrics returns](improve-prompts-guide.md#what-get-metrics-returns-and-which-values-decide). `ErrorRate` is `errors / Annotations` (it counts misses — not `1 - Precision`); the `Quality`/`ProjectScoreQuality` labels use inconsistent scales — never gate on them. | | `uip ixp projects configure-model [options] --output json` | Configure extraction model. Options: `--model` (gemini_2_5_flash/gemini_2_5_pro/gpt_4o_2024_05_13) and `--preprocessing` (none/table_mini/table). To read the current settings, see [Reading the current model and pre-processing](#reading-the-current-model-and-pre-processing). | | `uip ixp projects list-models --output json` | List all model versions and tags. Returns `Models[]` (`Version`, `ModelName`, `Pinned`, `TrainedTime`, `Description`), `Tags[]` (`Name`, `Version`, `UpdatedAt`), and `MaxPublished`. **The only read path for the project's live version** — `Tags[]` entry Name=`live`, else the highest `Models[]` with `Pinned: true`; which version a **folder** serves at runtime is a different question — [Deployments](#deployments). `ModelName` is the trained labeller's **family** (e.g. `gemini_ixp`, `gemini_pro_ixp`) — it is never a `--model` value like `gemini_2_5_flash`, so it does not answer "which extraction model is configured" (see [Reading the current model and pre-processing](#reading-the-current-model-and-pre-processing)). | | `uip ixp projects publish [--model-version ] [--tag ] --output json` | Publish a model version — defaults to the latest; pass `-m, --model-version ` to pick a specific one. `-d, --description ""` sets a description; `--tag ` tags the published version. | diff --git a/skills/uipath-ixp/references/improve-prompts-guide.md b/skills/uipath-ixp/references/improve-prompts-guide.md index 2adeaf58b1..6cbdc147a6 100644 --- a/skills/uipath-ixp/references/improve-prompts-guide.md +++ b/skills/uipath-ixp/references/improve-prompts-guide.md @@ -55,7 +55,7 @@ The values `get-metrics` returns are neither independent nor interchangeable — | `ErrorRate` | field, group | **Report — independent of `Precision`.** Wrong extractions over `Annotations`. A wrong value counts **once** (not as a false positive plus a false miss), and a miss counts even though it cannot lower `Precision` — so `Precision` 1.00 can still carry `ErrorRate` 0.20. Report it as the manual-correction burden; diagnose direction from `Precision`/`Recall`. | | `Quality` | field | **Ignore.** A coarse label derived from the numbers, on a scale inconsistent with `ProjectScoreQuality` (an `F1` of 1.00 still reads `good` while a `ProjectScore` of 0.91 reads `excellent`). Never gate on it and don't report it per field — if the user asks about the UI's label, explain the scales differ. | | `ProjectScoreQuality` | project | **Report on the project line only** (the label the UI shows beside the score) — different scale from field `Quality` (above). | -| `FieldGroup`, `FieldId`, `Name` | field | Identity. Compare on `FieldId` (stable); report on `Name` (the current display name — `null` for a deleted field, fall back to `FieldId`, see 1a). | +| `FieldGroup`, `FieldId`, `Name` | field | Identity. Compare on `FieldId` (stable); report on `Name` (that version's name for the field; on a rename, report the current version's — see 1a). | ## Waiting for retrain @@ -89,10 +89,9 @@ Note the `ModelVersion` from this baseline read — later iterations check that Save the full per-field `Fields` array as `baseline_metrics`. This is the starting point you compare against. (For a validated model, get-metrics Data is flat — `Fields`/`FieldGroups`/`ValidatedDocuments` are top-level. An unvalidated model returns `Data: { Metrics: null }` instead — re-fetch under the bounded wait above.) -**Field names:** each `Fields` entry carries both `FieldId` and `Name`, so report and compare fields straight from the metrics — do NOT fetch the taxonomy to build an id→name map. Three rules: +**Field names:** each `Fields` entry carries both `FieldId` and `Name`, so report and compare fields straight from the metrics — do NOT fetch the taxonomy to build an id→name map. Two rules: -- **Compare on `FieldId`, report on `Name`.** `FieldId` is stable; `Name` reflects the taxonomy as it is now, so a field renamed since an older version was scored reads back under its current name. -- **`Name` is null** when the service could not resolve it (e.g. the field was deleted after that version was scored). Fall back to `FieldId` — never skip the field. +- **Compare on `FieldId`, report on `Name`.** `FieldId` is stable. `Name` is that version's name for the field, so two versions can return different names for the same field — never join on `Name`. When they differ, report the current version's name. - **When two fields share a `Name`, qualify it with `FieldGroup`.** Display names are unique only *within* a group, so the same label can sit under two of them — print those rows as ` / ` or the reader cannot tell which one a score belongs to. This changes how you print the row, nothing else: the comparison still keys on `FieldId`. ### 1b. Check model configuration