docs(annotation): contributor README for adding locale YAML catalogs#210
Draft
henrycgbaker wants to merge 22 commits into
Draft
docs(annotation): contributor README for adding locale YAML catalogs#210henrycgbaker wants to merge 22 commits into
henrycgbaker wants to merge 22 commits into
Conversation
…aults New _InheritType private class + INHERIT singleton (Final[]) + PEP 695 type alias Inherit, mirroring the existing Unset pattern. Distinct from Unset: Unset marks "kwarg not provided" in ResolveSettings.resolve() overrides and is stripped by prune_unset before Pydantic.model_validate. Inherit means "inherit from parent scope" on nested settings models and persists through Pydantic validation; it will be replaced by the _propagate_cascade validator on AnnotationSettings (introduced in the next commit).
…ascade propagation Restructure AnnotationSettings so production_min_submitted and calibration_min_submitted can be overridden per workspace and per task via a cascade chain AnnotationSettings >= WorkspaceSettings >= TaskSettings. - Rename TaskOverlap -> TaskSettings; cascade fields now typed T | Inherit = INHERIT (default-free for cascade fields). - New WorkspaceSettings BaseModel with same cascade fields plus tasks: dict[Task, TaskSettings]. 1-to-N workspace->tasks preserved. - Rename AnnotationSettings.workspace_dataset_map -> workspaces: dict[str, WorkspaceSettings]. - Hoist production_min_submitted and calibration_min_submitted to top-level AnnotationSettings as opinionated deployment defaults (1 and 3). - New _propagate_cascade model_validator on AnnotationSettings: walks workspaces -> tasks and replaces INHERIT values with the parent scope's value at materialisation. Declared first among the mode="after" validators so downstream validators see post-propagation concrete values. - Update _check_calibration_topology to iterate self.workspaces.items() and use post-propagation per-task calibration_min_submitted; error message format changes from "tasks: [...]" to "(workspace, task) pairs: [...]". - Update _validate_task_uniqueness field paths (semantics unchanged). - Update consumers (record_builder, setup, export_fetcher, export_runner) to use the new shape. fan_out_records reads task_settings.* directly (post-cascade concrete values). assign_partitions call site unchanged -- calibration_fraction stays deployment-only because partitioning is record-level; per-workspace fractions would require a partition manifest refactor. - Update stale docstrings in argilla_task_definitions.py, cli/commands/annotation.py, api/annotation_setup.py. Hard YAML break: workspace_dataset_map -> workspaces with nested tasks: sub-key. No backwards-compatibility shims.
Migrate every test fixture from workspace_dataset_map/TaskOverlap to the new workspaces/WorkspaceSettings/TaskSettings shape across unit and integration tests. - Update _check_calibration_topology assertions to match the new "(workspace, task) pairs" error format. New unit-test classes in test_annotation_settings.py: - TestTaskSettingsDefaults, TestWorkspaceSettingsDefaults: default values, extra="forbid", calibration_min_submitted disable semantic. - TestCascadePropagation: workspace inherits from annotation, task inherits from workspace, explicit workspace/task overrides preserved, explicit None disable preserved through cascade, workspace None cascades to inheriting tasks, empty tasks dict no-op. - TestCalibrationTopologyValidator: per-(workspace, task) error format and cascade-aware detection. - TestTaskUniquenessValidator: same task across two workspaces rejected under new shape. - TestValidatorOrdering: behaviour-based pin that _propagate_cascade runs before _check_calibration_topology. - TestYamlRoundtrip: documents that model_dump() returns post-cascade concrete values (lossy round-trip is a documented non-goal). New integration test in test_annotation_calibration.py: - TestPerWorkspaceMinSubmitted:: test_workspace_carve_out_creates_correct_dataset_min_submitted - workspace + task carve-outs produce Argilla datasets with cascaded min_submitted (gated by live Argilla like other integration tests).
Mirror the in-file sentinel precedent: drop the unused singleton _instance/__new__ boilerplate (Pydantic uses isinstance, not identity, on the sentinel) and add the __bool__ footgun guard that _UnsetType already has.
- TestYamlRoundtrip: drop tmp-file write/read round-trip; yaml.safe_load on the string is enough to prove model_dump() returns post-cascade values. - test_error_message_reports_workspace_task_pairs: use pytest.raises ... as excinfo (matches the rest of the file) instead of try/except/else/pytest.fail.
…y findings
Naming: 'cascade' is the colloquial term but CSS-style 'inheritance' is the
technically correct one — child scope adopts the parent's value when no override
is set, exactly matching the existing INHERIT sentinel. Renames:
_propagate_cascade → _propagate_inheritance; _CASCADE_FIELDS → _INHERITED_FIELDS.
Simplifications:
- Extract _inherit(child, parent, fields) helper; collapses two near-identical
loops in _propagate_inheritance.
- Tighten TaskSettings / WorkspaceSettings docstrings (~13 → ~3 lines each);
move the inheritance-model explanation to the module docstring.
- Drop the leaky _InheritType import in record_builder.py; use typing.cast on
the post-validation value instead of forcing consumers to learn the private
sentinel.
- Clean up _check_calibration_topology error format: workspace/task pairs
rendered as 'retrieval/retrieval, ...' instead of '[("retrieval", "retrieval"), ...]'.
…ask() (CSS-style computed values)
…t strings
Introduces per-locale catalogs for Argilla dataset display strings (titles,
guidelines, label option text). EN catalog ships with this change; the Locale
enum reserves the DE slot for the next PR which adds the German catalog and
multi-locale discard widget rendering.
- New `Locale` StrEnum (EN, DE) in core/schemas/annotation_task.py
- `locale` field added to TaskSettings, WorkspaceSettings, AnnotationSettings
with cascade propagation through `_CASCADE_FIELDS`
- Catalog scaffolding: `core/annotation/locales/{__init__, types, en}.py`
keyed by `(task, kind, name)` with kinds field/question/guidelines/label
- `argilla_task_definitions.build_task_settings(locale)` parameterised on
locale; field/question/guidelines titles and `LabelQuestion(labels=...)`
dicts all driven by the catalog
- Identities (`name=`) and label values (`yes`/`no`/`DiscardReason.*.value`)
remain stable across locales so exports merge cleanly
…hten catalog typing
Locale becomes an open `str` type alias (was a closed StrEnum) so a deployment can add a locale without editing pragmata source. Translations move from `en.py` (Python dict with imported enum keys) to `en.yaml` (data). The registry auto-discovers `*.yaml` files at import time, keyed by filename stem. The locale-invariant structure (which questions share the yes/no LabelQuestion, the DiscardReason set) stays in code as `loader.py`'s `_YES_NO_QUESTIONS_BY_TASK` constant; the loader fans the YAML inputs over that structure into the flat tuple-keyed Catalog shape consumers expect. Adding a locale is now: drop `<code>.yaml` into `core/annotation/locales/`. No Python edit required.
…on loop Replace the inline `yaml.safe_load(path.read_text())` in `load_catalog` with the project's existing `load_config_file` helper, which already handles path coercion, `expanduser`, `FileNotFoundError`, the `None`-doc case, and non-mapping-root validation. Consistent with every other YAML consumer in the repo. Iterate `DiscardReason` for the discard-reason label fan-out instead of iterating the YAML dict — a missing or typo'd reason in en.yaml now raises `KeyError` at load, matching the loader's fail-loud contract. Also collapse the field/question loops (only the `CatalogKind` differs).
Extends the locale catalog system with a German (DE) catalog and turns the discard widget into an i18n-aware template that ships every supported locale and switches at runtime via Argilla's chrome-locale toggle. - New DE catalog (locales/de.py) mirroring the EN key set - EN catalog gains widget chrome strings (panel summary, button label, etc.) - Catalog completeness test guarantees non-EN locales define exactly the keys EN provides; spot-checks for translated label displays and value stability - discard_flow.html: i18n JSON payload + supported-locales array injected at render time; JS picks active locale via $nuxt.$i18n.onLanguageSwitched with a chrome → html-lang → dataset-creation → EN fallback chain - _render_discard_template / _discard_i18n_payload_for_locale in argilla_task_definitions.py do the substitution per (task, dataset_locale) - Existing HTML-option enum-sync test rewritten against the rendered payload Identities (`name=`) and label values (`yes`/`no`/`DiscardReason.*.value`) remain stable across locales — exports merge cleanly across multi-language deployments.
… guard, DE phrasing
…lict warning Wires the cascade-resolved locale through the import pipeline so each workspace's datasets render in its declared locale, and adds a CLI flag plus a re-import safety check. - `--locale` flag on `import` only (NOT on setup/teardown — locale only matters at dataset creation) - `api.annotation_import` accepts a `locale` kwarg that overrides the deployment default through the existing settings cascade - `record_builder.fan_out_records` reads `task_settings.locale` per dataset - `record_builder._ensure_dataset` detects locale mismatch on re-import by inspecting the existing dataset's `LabelQuestion` option keys against `CATALOGS`; logs a warning and appends — data integrity preserved because label values are locale-invariant, only display strings are frozen at dataset creation
Now that Locale is an open str type alias (not StrEnum): - parse_locale validates against the registry rather than calling Locale(raw), raising ValueError with a "Supported: ..." hint on unknown locales. - record_builder's locale-mismatch warning drops `.value` and `is` identity in favour of equality; existing_locale and locale are both plain strings. Tests follow: assertions compare against "en"/"de" string literals; the Locale enum import is dropped.
…ed import flags Wire two additional flags through the import settings inheritance chain: - `--no-calibration` disables calibration in one shot by composing `calibration_min_submitted=None` and `calibration_fraction=0.0`. Conflicts with `--calibration-fraction > 0` and exits with code 2. - `--calibration-partition-seed INT` overrides the deployment-level partition seed; existing assignments remain locked by the manifest. `api.annotation_import.import_records` gains matching `calibration_min_submitted` and `calibration_partition_seed` kwargs (typed `T | Unset = UNSET`), threaded into the settings overrides dict.
Replace 'float | object' / 'int | None | object' fallback annotations with the proper 'float | Unset' / 'int | None | Unset' types already used in the API layer. Re-export Unset from the api facade so cli/ can reference the type without crossing the cli -> core boundary.
Documents the locales/ package layout (types / structure / registry / per-locale catalogs), the catalog key shape, the cross-locale stability invariant, and a 4-step guide for adding a new locale that reflects the current module split.
github-actions
Bot
added
documentation
henrycgbaker
force-pushed
the
docs/annotation-locales-readme
branch
3 times, most recently
from
15c7211 to
df35312
Compare
…ocale Catalogs are now YAML data (en.yaml, de.yaml) auto-discovered by the registry, not Python modules. Locale is an open str type, not a closed StrEnum. Adding a locale is "drop a YAML file" — no enum edit, no registry edit, no import statement. The 4-step Python guide is replaced with a 3-step data guide (copy YAML, translate values, run tests). The correctness invariant section now points at loader.py instead of structure.py and adds DiscardReason as a third invariant the YAML must keep in sync.
henrycgbaker
force-pushed
the
docs/annotation-locales-readme
branch
from
df35312 to
3be998c
Compare
Collaborator
Author
|
@saschagobel I've put this together for now as quick first pass, but it's low priority and i think should wait until we've had full discussion about documentation etc, so leaving in draft for now. Can also close & leave branch up if preferred |
henrycgbaker
force-pushed
the
feat/annotation-cli-inheritance-flags
branch
3 times, most recently
from
7ef68bd to
4d2b62e
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
DEFERRED FOR FULL DOCS DISCUSSION
Stack (6 PRs): #206 → #196 → #207 → #208 → #209 → this PR
Chain goal: Locale-aware Argilla datasets (EN/DE) on top of a deployment/workspace/task settings inheritance hierarchy, with live widget locale switching and CLI flags for per-import overrides.
This PR (6/6): Contributor README for
core/annotation/locales/.Goal
Document the locales package so a contributor (translator or dev) can add a new locale without reading the source. Reflects the YAML-as-data + open-
Locale-string layout introduced in #196 and the discard-widget chrome strings introduced in #207.Scope
src/pragmata/core/annotation/locales/README.md— single file, sits next to the YAML catalogs it documents.Covers:
types.py/loader.py/registry.py/<code>.yaml)_YES_NO_QUESTIONS_BY_TASK,DISCARD_WIDGET_KEYS,DiscardReason)Why this beats the old version
The old README documented the Python-as-data flow: edit the
Localeenum, write a<locale>.pymodule callingbuild_programmatic_entries(), register the catalog inregistry.py. That's three Python edits plus a YAML-free workflow that excluded non-Python translators.After #196 the flow is: drop a YAML file. Zero Python edits. The README now matches.
Test plan
uv run ruff check src testsclean (markdown-only change; no code touched)KeyErrorat load