Repository navigation
feat: Add Attribute Rules #32
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from all commits
Commits
Show all changes
12 commits
Select commit
Hold shift + click to select a range
38d78cf
chore: add feature updates and resolve conflicts
acneville cc93bd5
chore: remove unnecessary prompt engineering comment for dot ownership
acneville dc2dee5
chore: apply labelPoint to Centroid call
acneville 8d824aa
refactor: no longer need new_ar.py
acneville 3c339cb
refactor: moving county computation to within conditional to save on …
acneville 4ca9a5e
refactor: formatting
acneville 94c1324
refactor: fix arcade indent level
acneville 5dbc444
docs: README refactor
acneville f23d240
fix: repaired to proper midpoint computation
acneville 97d4c8e
refactor: AR combination
acneville ed39a37
fix: unit test failure fixed
acneville 6010aac
Merge branch 'main' into feat/attribute-rules
acneville File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,24 @@ | ||
| version: 2 | ||
| updates: | ||
| - package-ecosystem: npm | ||
| directory: /attribute-rules | ||
| schedule: | ||
| interval: quarterly | ||
| groups: | ||
| safe-dependencies: | ||
| update-types: | ||
| - minor | ||
| - patch | ||
| major-dependencies: | ||
| update-types: | ||
| - major | ||
| commit-message: | ||
| prefix: deps | ||
| prefix-development: deps(dev) | ||
| cooldown: | ||
| default-days: 10 | ||
| semver-major-days: 60 | ||
| semver-minor-days: 14 | ||
| semver-patch-days: 7 | ||
| exclude: | ||
| - '@ugrc/*' |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,39 @@ | ||
| name: Pull Request Events (Attribute Rules) | ||
|
|
||
| on: | ||
| pull_request: | ||
| paths: | ||
| - "attribute-rules/**/*" | ||
|
|
||
| concurrency: | ||
| group: ${{ github.workflow }}-${{ github.head_ref || github.ref }} | ||
| cancel-in-progress: true | ||
|
|
||
| jobs: | ||
| check: | ||
| name: Test Arcade Expressions | ||
| runs-on: ubuntu-latest | ||
|
|
||
| steps: | ||
| - name: ⬇️ Set up code | ||
| uses: actions/checkout@v7 | ||
|
|
||
| - name: 📦 Setup PNPM | ||
| uses: pnpm/action-setup@v6 | ||
| with: | ||
| version: 11 | ||
|
|
||
| - name: ⎔ Setup Node.js | ||
| uses: actions/setup-node@v7 | ||
| with: | ||
| node-version: lts/* | ||
| cache: pnpm | ||
| cache-dependency-path: attribute-rules/pnpm-lock.yaml | ||
|
|
||
| - name: 📥 Download dependencies | ||
| working-directory: attribute-rules | ||
| run: pnpm install --frozen-lockfile | ||
|
|
||
| - name: 🧪 Run tests | ||
| working-directory: attribute-rules | ||
| run: pnpm test |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
acneville marked this conversation as resolved.
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1 @@ | ||
| /node_modules |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,71 @@ | ||
| # Attribute Rules | ||
|
|
||
| The attribute-rule utilities are in [deploy_ar.py](deploy_ar.py) and [deploy_all_ar.py](deploy_all_ar.py). | ||
|
|
||
| - [deploy_ar.py](deploy_ar.py) is the primary script. | ||
| - Default mode is idempotent deploy: add if missing, alter if mutable changes, recreate when immutable changes require it. | ||
| - Use `--mode add` for strict add-only behavior. | ||
| - Use `--mode alter` to update an existing rule only. | ||
| - [deploy_all_ar.py](deploy_all_ar.py) recursively deploys every `.arcade` file using idempotent deploy mode. It infers the rule type from the top-level folder (`calculation`, `constraint`, or `validation`) and reads optional `// KEY: value` metadata from each script. | ||
|
|
||
| Supported per-script metadata keys are `RULE_NAME`, `FIELD`, `EVENTS`, `IS_EDITABLE`, `ERROR_NUMBER`, `ERROR_MESSAGE`, `DESCRIPTION`, and `SUBTYPE`. Use `FIELD:` with no value for calculation rules that return an `attributes` dictionary for multiple fields. | ||
|
|
||
| Examples: | ||
|
|
||
| ```bash | ||
| # deploy mode (default) | ||
| python attribute-rules/deploy_ar.py fullname_calculation --in-table "Z:/data/County.gdb/Roads" --field FULLNAME | ||
|
|
||
| # add-only behavior | ||
| python attribute-rules/deploy_ar.py fullname_calculation --in-table "Z:/data/County.gdb/Roads" --mode add --field FULLNAME | ||
|
|
||
| # alter existing rule only | ||
| python attribute-rules/deploy_ar.py fullname_calculation --in-table "Z:/data/County.gdb/Roads" --mode alter --field FULLNAME | ||
|
|
||
| # recursively deploy every rule using its script metadata | ||
| python attribute-rules/deploy_all_ar.py --in-table "Z:/data/County.gdb/Roads" | ||
|
|
||
| # preview all actions without changing the geodatabase | ||
| python attribute-rules/deploy_all_ar.py --in-table "Z:/data/County.gdb/Roads" --dry-run | ||
| ``` | ||
|
|
||
| ## Deploying to an enterprise geodatabase | ||
|
|
||
| Attribute rule tools require a file, mobile, or enterprise geodatabase dataset. A feature service URL such as `https://server/server/rest/services/Roads/FeatureServer/0` cannot be used as `--in-table`, and ArcGIS Online hosted feature layers are not a supported target. Deploy to the enterprise geodatabase instead. | ||
|
|
||
| The UTRANS enterprise geodatabase is SQL Server on `db.ugrc.utah.gov`. Create a connection file once, then use its `.sde` path in place of a local `.gdb` path. | ||
|
|
||
| Option 1 — ArcGIS Pro: use the **New Database Connection** dialog in the Catalog pane (Database Platform `SQL Server`, Instance `db.ugrc.utah.gov`, Authentication Type `Database authentication`, User name `TRANSADMIN`). Then right-click the finished connection and choose **Copy Path** to get the `.sde` location to pass to the script. Connections saved to Catalog favorites live in `%APPDATA%\Esri\ArcGISPro\Favorites`, so you can also browse to the file directly. | ||
|
|
||
| Option 2 — Python: | ||
|
|
||
| ```python | ||
| import arcpy | ||
|
|
||
| arcpy.management.CreateDatabaseConnection( | ||
| out_folder_path="Z:/connections", | ||
| out_name="roadtest_transadmin.sde", | ||
| database_platform="SQL_SERVER", | ||
| instance="db.ugrc.utah.gov", | ||
| account_authentication="DATABASE_AUTH", | ||
| username="TRANSADMIN", | ||
| save_user_pass="SAVE_USERNAME", | ||
| database="RoadTest", | ||
| ) | ||
| ``` | ||
|
|
||
| Append the dataset name to the connection path and deploy: | ||
|
|
||
| ```bash | ||
| python attribute-rules/deploy_all_ar.py --in-table "Z:/connections/roadtest.sde/TRANSADMIN.Roads_Edit" --dry-run | ||
| ``` | ||
|
|
||
| Requirements to be aware of: | ||
|
|
||
| - Connect as the dataset owner (`TRANSADMIN`). Attribute rule changes made by any other account fail validation. | ||
| - The identity that matters is the database login stored in the `.sde` file, not your Windows account or ArcGIS Pro portal sign-in. Check it with `arcpy.Describe("<connection>.sde").connectionProperties.user`. | ||
| - Validation rules require branch versioning on an enterprise geodatabase. Calculation and constraint rules do not. | ||
| - Datasets are qualified as `schema.table`. Feature classes inside a feature dataset can also be addressed as `<connection>.sde/TRANSADMIN.Centerlines_Edit/TRANSADMIN.Roads_Edit`. | ||
| - Always run `--dry-run` first. It previews each rule action and warns about owner and versioning problems, but because it never calls the geoprocessing tool it cannot catch every server-side rejection. | ||
| - Confirm a connection's platform with `arcpy.Describe("<connection>.sde").connectionProperties.dbclient`. | ||
| - Keep connection files outside the repository so database credentials are never committed. |
26 changes: 26 additions & 0 deletions
26
attribute-rules/calculation/Roads_Edit/convert_to_null_if_no_value.arcade
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,26 @@ | ||
| /* global $feature, $datastore, $layer, $map, FeatureSetByName, IIf, Buffer */ | ||
| // ADDRSYS_L/R are text fields under CVDomain_AddressReferenceSystem, so null is their only valid empty value. | ||
| // FIELD: | ||
| // EVENTS: INSERT, UPDATE | ||
| var textFields = ['ADDRSYS_L', 'ADDRSYS_R', 'BIKE_L', 'BIKE_R', 'BIKE_PLN_L', 'BIKE_PLN_R', 'DOT_HWYNAM', 'DOT_RTNAME', 'DOT_RTPART']; | ||
| Expects($feature, 'ADDRSYS_L', 'ADDRSYS_R', 'BIKE_L', 'BIKE_R', 'BIKE_PLN_L', 'BIKE_PLN_R', 'DOT_HWYNAM', 'DOT_RTNAME', 'DOT_RTPART'); | ||
| var updates = {}; | ||
|
|
||
| for (var i in textFields) { | ||
| var f = textFields[i]; | ||
| var val = $feature[f]; | ||
|
|
||
| // Convert empty/whitespace strings to null, leave valid values alone | ||
| if (IsEmpty(Trim(val))) { | ||
| updates[f] = null; | ||
| } else { | ||
| updates[f] = val; | ||
| } | ||
| } | ||
|
|
||
| // For calculation rules, "attributes" must be nested under "result". | ||
| return { | ||
| "result": { | ||
| "attributes": updates | ||
| } | ||
| }; |
28 changes: 28 additions & 0 deletions
28
attribute-rules/calculation/Roads_Edit/fullname_calculation.arcade
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,28 @@ | ||
| /* global $feature, $datastore, $layer, $map, FeatureSetByName, IIf, Buffer */ | ||
| // FIELD: FULLNAME | ||
| // EVENTS: INSERT, UPDATE | ||
| function normalizePart(value) { | ||
| var part = Trim(IIf(IsEmpty(value), "", Text(value))); | ||
| while (Find(" ", part) > -1) { | ||
| part = Replace(part, " ", " "); | ||
| } | ||
| return part; | ||
| } | ||
|
|
||
| var name = normalizePart(Replace(IIf(IsEmpty($feature.NAME), "", $feature.NAME), "'", "")); | ||
| var postdir = normalizePart($feature.POSTDIR); | ||
| var posttype = normalizePart($feature.POSTTYPE); | ||
|
|
||
| var parts = []; | ||
| if (!IsEmpty(name)) { | ||
| Push(parts, name); | ||
| } | ||
| if (!IsEmpty(postdir)) { | ||
| Push(parts, postdir); | ||
| } | ||
| if (!IsEmpty(posttype)) { | ||
| Push(parts, posttype); | ||
| } | ||
|
|
||
| // Test case: NAME=" MAIN ", POSTDIR=null, POSTTYPE=" RD " -> "MAIN RD" | ||
| return Concatenate(parts, " "); |
7 changes: 7 additions & 0 deletions
7
attribute-rules/calculation/Roads_Edit/remove_apostrophes_from_name.arcade
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,7 @@ | ||
| /* global $feature, $datastore, $layer, $map, FeatureSetByName, IIf, Buffer */ | ||
| // FIELD: NAME | ||
| // EVENTS: INSERT, UPDATE | ||
| // Remove apostrophes from NAME field on CREATE and UPDATE events | ||
| // Test case: NAME="O'CONNELL" -> "OCONNELL" | ||
| var name = IIf(IsEmpty($feature.NAME), "", Text($feature.NAME)); | ||
| return Replace(name, "'", ""); |
12 changes: 12 additions & 0 deletions
12
attribute-rules/calculation/Roads_Edit/remove_extra_whitespace.arcade
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,12 @@ | ||
| /* global $feature */ | ||
| // FIELD: NAME | ||
| // EVENTS: INSERT, UPDATE | ||
| // Trim NAME and collapse accidental internal whitespace. | ||
| // Test case: NAME=" NORTH MAIN " -> "NORTH MAIN" | ||
|
|
||
| var value = Trim(IIf(IsEmpty($feature.NAME), "", Text($feature.NAME))); | ||
| while (Find(" ", value) > -1) { | ||
| value = Replace(value, " ", " "); | ||
| } | ||
|
|
||
| return value; |
73 changes: 73 additions & 0 deletions
73
attribute-rules/calculation/Roads_Edit/spatial_assignments.arcade
|
acneville marked this conversation as resolved.
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,73 @@ | ||
| /* global $feature, $datastore, $layer, $map, FeatureSetByName, IIf, Buffer */ | ||
| // FIELD: | ||
| // EVENTS: INSERT, UPDATE | ||
| Expects($feature, 'DOT_RTNAME'); | ||
|
|
||
| var addressSystemQuadrantsLayer = FeatureSetByName($datastore, "AddressSystemQuadrants", ["QUADRANT", "GRID_NAME"]); | ||
| var countyLayer = FeatureSetByName($datastore, "Counties", ["FIPS_STR", "NAME"]); | ||
| var zipCodeLayer = FeatureSetByName($datastore, "ZipCodes", ["ZIP5", "NAME"]); | ||
| var municipalityLayer = FeatureSetByName($datastore, "Municipalities", ["NAME"]); | ||
|
|
||
| var roadLeft = Centroid(Offset($feature, -15, 'meters', 'square'), 'labelPoint'); | ||
| var roadRight = Centroid(Offset($feature, 15, 'meters', 'square'), 'labelPoint'); | ||
|
|
||
| function fieldOrNull(feature, fieldName) { | ||
| if (IsEmpty(feature)) { | ||
| return null; | ||
| } | ||
|
|
||
| return feature[fieldName]; | ||
| } | ||
|
|
||
| function dotOwner(municipalityFeature, countyFeature) { | ||
| if (!IsEmpty(municipalityFeature)) { | ||
| return 'MU ' + municipalityFeature.NAME; | ||
| } | ||
|
|
||
| if (!IsEmpty(countyFeature)) { | ||
| return 'CO ' + Proper(countyFeature.NAME); | ||
| } | ||
|
|
||
| return ''; | ||
| } | ||
|
|
||
| var quadrantLeftFeature = First(Intersects(addressSystemQuadrantsLayer, roadLeft)); | ||
| var quadrantRightFeature = First(Intersects(addressSystemQuadrantsLayer, roadRight)); | ||
| var countyLeftFeature = First(Intersects(countyLayer, roadLeft)); | ||
| var countyRightFeature = First(Intersects(countyLayer, roadRight)); | ||
| var zipCodeLeftFeature = First(Intersects(zipCodeLayer, roadLeft)); | ||
| var zipCodeRightFeature = First(Intersects(zipCodeLayer, roadRight)); | ||
| var municipalityLeftFeature = First(Intersects(municipalityLayer, roadLeft)); | ||
| var municipalityRightFeature = First(Intersects(municipalityLayer, roadRight)); | ||
|
|
||
| var dotRtName = Trim(IIf(IsEmpty($feature.DOT_RTNAME), '', Text($feature.DOT_RTNAME))); | ||
| var dotOwnL = ''; | ||
| var dotOwnR = ''; | ||
| if (Left(dotRtName, 1) == '0') { | ||
| dotOwnL = 'UDOT'; | ||
| dotOwnR = 'UDOT'; | ||
| } else { | ||
| dotOwnL = dotOwner(municipalityLeftFeature, countyLeftFeature); | ||
| dotOwnR = dotOwner(municipalityRightFeature, countyRightFeature); | ||
| } | ||
|
|
||
| return { | ||
| "result": { | ||
| "attributes": { | ||
| "QUADRANT_L": fieldOrNull(quadrantLeftFeature, "QUADRANT"), | ||
| "QUADRANT_R": fieldOrNull(quadrantRightFeature, "QUADRANT"), | ||
| "COUNTY_L": fieldOrNull(countyLeftFeature, "FIPS_STR"), | ||
| "COUNTY_R": fieldOrNull(countyRightFeature, "FIPS_STR"), | ||
| "ADDRSYS_L": fieldOrNull(quadrantLeftFeature, "GRID_NAME"), | ||
| "ADDRSYS_R": fieldOrNull(quadrantRightFeature, "GRID_NAME"), | ||
| "POSTCOMM_L": fieldOrNull(zipCodeLeftFeature, "NAME"), | ||
| "POSTCOMM_R": fieldOrNull(zipCodeRightFeature, "NAME"), | ||
| "ZIPCODE_L": fieldOrNull(zipCodeLeftFeature, "ZIP5"), | ||
| "ZIPCODE_R": fieldOrNull(zipCodeRightFeature, "ZIP5"), | ||
| "INCMUNI_L": fieldOrNull(municipalityLeftFeature, "NAME"), | ||
| "INCMUNI_R": fieldOrNull(municipalityRightFeature, "NAME"), | ||
| "DOT_OWN_L": dotOwnL, | ||
| "DOT_OWN_R": dotOwnR | ||
| } | ||
| } | ||
| } |
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.