Repository navigation
Conversation
Copied from the GeoPressure/GeoLocator-DP v1.1-resource-schema branch. Re-sync with data-raw/sync_schemas.R once v1.1 is tagged. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
v1.1 moves the table schema from `resource$$schema` to `resource$schema` and requires `type`. Resources are rebuilt against the v1.1 schemas by normalize_upgraded_resources(), so the step function only bumps the profile URL. Validating real output against the v1.1 profile gives 0 errors on both frictionless 1.3.0 and 2.0, where v1.0 gave 8 failing resources. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Merged
Codecov Report❌ Patch coverage is
📢 Thoughts on this report? Let us know! |
GeoLocator-DP v1.1 selects the resource schema with `if`/`then` on the resource name instead of `oneOf`, so the four places that read the resource names off the profile stopped finding them. `upgrade_gldp()` walks packages written against any earlier version, so both shapes have to be read, not just the current one. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Picks up the string `fieldsMatch`, the array primary and foreign keys, and the restructured profile. Copied from the GeoLocator-DP `v1.1-resource-schema` branch; to be re-synced with `sync_gldp_schemas()` once v1.1 is tagged. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
GeoLocator-DP v1.1 declares `categories` next to the `enum` constraint. Both restrict a field to a fixed set of values, so read whichever the schema gives: a schema that drops `enum` later is then still validated. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Picks up `categories` and the two corrected examples. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
GeoLocator-DP v1.1 requires `$schema` on its tabular resources, and frictionless only sets it from version 2.0. Set it here, alongside `type`, so packages are correct on both. `profile: "tabular-data-resource"` stays for now even though `$schema` and `type` replace it: frictionless 1.3.0 aborts with `Resource "tags" must have a profile property with value` when it is absent, and that is the release on CRAN. It can go once the dependency moves to frictionless 2.0. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
GeoLocator-DP moved `tags` from `fieldsMatch: superset` to `partial`, so a package may carry extra attributes on a tag. `add_gldp_resource()` already handles both modes and now keeps those columns instead of dropping them with a warning. The test covering the dropping path moves to `staps`, which still declares `superset`, and `tags` gains one covering the keeping path. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
`frictionless::read_resource()` maps schema fields to columns by position, because `fieldsMatch` is not implemented (frictionlessdata/frictionless-r#216). A file that leaves out a column its schema declares is therefore read with every later column under the wrong name, silently: a `tags.csv` holding only `tag_id` and `ring_number` came back with the ring number labelled `datapackage_id`, the second field the schema declares. Nothing caught it. `validate_gldp()` only ever sees the names frictionless assigned, and `{tag_id, datapackage_id}` is a legal subset of the declared fields, so it reported the table as consistent with its schema. Every GeoLocator-DP table declares `superset` or `partial`, both of which let a file leave columns out, so this is a file we tell people they may write. The schema is now reconciled with the header before reading: declared fields absent from the file are dropped, the rest are put in file order, and columns the schema does not describe are either described as `any`, where `fieldsMatch` allows them, or reported by name rather than through frictionless's "Invalid input type, expected 'list' actual 'NULL'". Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
`fieldsMatch` is a rule over the whole set of fields and carries no notion of which ones are required, so both modes GeoLocator-DP uses let a table leave out a column declaring `required: true`. A `tags` table with no `tag_id` at all validated clean, because `required` was only ever checked for columns that were present. The standard cannot yet express "the required columns must be present, the optional ones may be absent" (frictionlessdata/datapackage#1126), so the presence of required columns is checked here instead. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Supports GeoLocator-DP v1.1 and makes it the default for
create_gldp().v1.1 moves the table schema from
resource.$schematoresource.schemaand requires$schemaandtype. Validating real GeoLocatoR output against the published profile:First time the output conforms to its own profile.
Changes
inst/schemas/v1.1/— the profile and 8 table schemas, synced from the tag."v1.1"added to.gldp_supported_versions, making it the default.upgrade_gldp_v1_0_to_v1_1()only bumps the profile URL;normalize_upgraded_resources()already rebuilds resources against the target schemas.if/thenwhere v1.0 usedoneOf.add_gldp_resource()sets$schemaandtype: "table"itself, since frictionless only writes them from version 2.0.profilestays: frictionless 1.3.0 won't read a resource without it.categoriesas well asenum, v1.1 declaring both.Two bugs found along the way
read_resource()maps schema fields to columns by position,fieldsMatchbeing unimplemented (frictionless-r#216). Atags.csvholding onlytag_idandring_numberread the ring number asdatapackage_id, silently — andvalidate_gldp()passed it, since it only sees the names frictionless assigned. Every table allows omitted columns, so this is a file we tell people they may write. Reads now match the schema to the CSV header by name.fieldsMatchsays nothing about which columns are required, so atagstable with notag_idvalidated clean. The standard can't yet express this (datapackage#1126), sovalidate_gldp()checks it directly.Verification
387 tests pass on both frictionless 1.3.0 and 1.3.0.9000 (the 49 warnings are pre-existing — same count with these changes stashed). Emitted packages validate against the published v1.1 profile with 0 errors and report
version(resource)as2.0. That covers #38 for this repo: both versions work, so users need change nothing.Release-time work not included here:
DESCRIPTION/CITATION.cffversions and theNEWS.mdblock. Gated on frictionless 2.0 reaching CRAN: droppingprofile, removing the version-warning muffle inread_gldp(), and raising theDESCRIPTIONfloor.🤖 Generated with Claude Code