objex-utils package split, runtime config + settings panel, deck.gl-raster 0.7, TS 6#15
Merged
Conversation
…relocation Sets up pre-flight infra for moving framework-agnostic utilities from src/lib/utils/ to packages/objex-utils/src/: - scripts/verify-objex-utils-bundle.mjs scans the tsup dist for any top-level import of forbidden heavy deps (@developmentseed/*, proj4, wkt-parser, maplibre-gl, luma.gl, deck.gl, etc.). Wired into the package build script so the guardrail runs on every build, locking in the cog-pure.ts pattern that resolved #11. - Root package.json now depends on @walkthru-earth/objex-utils via workspace:* so src/lib/utils/<name>.ts shim files can re-export from the sibling package without breaking the published @walkthru-earth/objex tarball. - Changeset opened as a single patch/patch bump that the relocation batches will piggyback on.
…ls/src Moves wkb, hex, format, column-types, file-sort, error, lru, connection-identity, storage-smoketest, stac-storage-extension, cog-pure, channel-composite. Each src/lib/utils/<name>.ts is now a thin shim re-exporting from @walkthru-earth/objex-utils, so existing intra-app imports continue to resolve. Bundle guardrail passes: no heavy-dep top-level imports introduced.
Moves geoarrow, parquet-metadata, geometry-type, storage-url, cloud-url. Old src/lib/utils/<name>.ts files become shims re-exporting from @walkthru-earth/objex-utils. Bundle guardrail passes: apache-arrow / hyparquet are declared peers, no other heavy deps introduced.
Integrates Batch C (stac, stac-source, stac-facets, stac-pushdown, stac-geoparquet, cog-asset, markdown-sql, host-detection) and finishes the cross-batch fixes that surfaced once all three worktrees merged: - Rewrite three intra-package imports that were routing through src/lib/utils/ shims back into the npm package: channel-composite, host-detection, and stac-storage-extension now use sibling ./*.js paths to avoid a circular dependency through @walkthru-earth/objex-utils. - Revert Batch C's tsup external addition of @walkthru-earth/objex-utils. Marking the package as self-external is exactly the anti-pattern from #11: tsup would emit a bare self-import in dist, Vite consumers would fail to pre-bundle it. The intra-package import rewrites above remove the need for that workaround. - Move src/lib/utils/export.ts into packages/objex-utils/src/export.ts. The previous batch had patched export.ts to import jsonReplacerBigInt via a cross-package relative path (../../../packages/objex-utils/src/format.js), which would survive svelte-package and ship a broken path inside the published @walkthru-earth/objex tarball. Moving export.ts into the package restores a clean ./format.js sibling import. - Rewrite packages/objex-utils/src/index.ts using export * for every intra-package module so the public surface stays in lockstep with each file's exports. Two remaining src/lib/utils/ re-exports (local-storage) keep their explicit named form. Verification: build (incl. guardrail), format, lint:fix, check, package (svelte-package + publint), deadcode all green. No $lib/ or cross-package relative paths appear in either dist/.
Remove the thin re-export shim files left in src/lib/utils/ after the utilities moved into @walkthru-earth/objex-utils, rewrite all in-repo imports to pull from the package directly, and drop the invalid ./utils/* subpath exports from the objex package.json exports map. No runtime behavior changes.
…ql-context with injected engine
Sweep the package's own docs/ Source pointers and the deep-dive reference docs (wkb-to-geoarrow, mosaic-layer-stac) so they cite the relocated packages/objex-utils/src/ paths instead of the removed src/lib/utils/ files. Leaves historical design-proposal references (multicog.ts, mosaic.ts, fetch-pool.ts) and valid app-side modules (cog.ts, zarr.ts) untouched.
… CORS-aware file tree - Relabel minio provider to 'MinIO / RustFS / Custom' for self-hosted / S3-compatible stores (MinIO, RustFS, Ceph RGW, custom endpoints). id unchanged so connections, CORS/read-only help, and host detection keep working. - Extract shared buildHttpsUrlForConnection in signed-url.ts; FileTreeSidebar 'Copy HTTP URL' no longer emits an AWS URL for GCS and other non-AWS connections (including empty-endpoint auto-detected GCS). - Surface a CORS/network failure in the file tree instead of a misleading 'Empty bucket' state; new fileTree.cors* / loadError* i18n keys (en + ar). Addresses #14
- Update ~25 deps via pnpm update (SvelteKit 2.61, Svelte 5.55.10, DuckDB-WASM dev53→dev55, Biome 2.4.16, hyparquet, mermaid, @Milkdown, @chunkd, etc.) - Migrate @geoarrow/deck.gl-layers@0.3.2 → @geoarrow/deck.gl-geoarrow@0.4.1 (renamed package). 0.4+ layers take a single Arrow RecordBatch as `data` instead of a Table; deck.ts now passes table.batches[0] (buildSingleTable emits one batch, so row index and sourceIndices click mapping are unchanged) - Move pnpm overrides + patchedDependencies from the deprecated package.json `pnpm` field into pnpm-workspace.yaml (pnpm v10 ignores the package.json field). Drop the stale 0.6.0-alpha.1 patch reference; pin the real 0.6.1 geotiff/raster + carbonplan zarr-layer patches - Update CLAUDE.md/docs references for the renamed geoarrow package
Bump all six @developmentseed/* packages (deck.gl-raster, deck.gl-geotiff, deck.gl-zarr, epsg, proj, geotiff) from 0.6.1 to 0.7.0 and slim the pnpm patch set now that upstream ships the behavior by default. Patches removed: - deck.gl-raster patch deleted entirely. PR #546 forwards onTileLoad/ onTileError/onTileUnload/onViewportLoad through RasterTileLayer natively - MosaicLayer callback + debounceTime hunk dropped from the geotiff patch. 0.7.0 forwards debounceTime natively and exposes source-level callbacks (onSourceLoad/onSourceError/onSourceUnload/onViewportLoad) Patch kept, regenerated against 0.7.0 dist hashes (@developmentseed__deck.gl-geotiff@0.7.0.patch), three hunks upstream still does not ship: proj4 +over antimeridian fix, inferRenderPipeline re-export, and the r16unorm to r32float Firefox/macOS texture fallback. Code migrations for 0.7.0 breaking changes: - StacMosaicViewer: onTileUnload -> onSourceUnload on MosaicLayer - cog.ts: MinimalTileData now imports from deck.gl-raster (no longer re-exported by deck.gl-geotiff) - TilesetDescriptor rename and MosaicTileset2D privatization need no changes (we import neither) Docs, patch-reference comments, and a changeset updated. check passes 0 errors, both packages build clean.
…match Code scanning (CodeQL js/polynomial-redos), all 8 High alerts: - connection-identity, storage-url: replace `/\/+$/` and `/^\/+|\/+$/g` slash-trim regexes with linear char-scan helpers - cloud-url: rewrite `([^/]+)\/?(.*)` to `([^/]+)(?:\/(.*))?` to remove the [^/]+ / .* backtracking ambiguity on s3:// and gcs:// URLs - markdown-sql: use `[ \t]` instead of `\s` around the sql-fence name so the trailing `\n` is an unambiguous boundary - column-types: strip the parenthesized type span via indexOf/lastIndexOf slicing instead of `/\(.*\)/` Behavior preserved (verified against the built bundle); 100k-char malicious inputs now run sub-millisecond instead of O(n^2). Dependabot transitive alerts via scoped pnpm overrides: - cookie -> ^0.7.0 (resolves 0.7.2, GHSA-pxg6-pf52-xh8x) - picomatch@4 -> ^4.0.4 (resolves 4.0.4, GHSA-3v7f-55p6-f55p), scoped to 4.x so micromatch's picomatch@2.x is untouched
…le when rail is hidden
The docs index linked 8 pages that were never created (stac, stac-source, stac-hydrate, stac-storage-extension, cog-asset, channel-composite, map-pixel-inspect, lru) and omitted the new app-config module entirely. Those broken links shipped in the npm tarball since docs/ is in files. - Add reference pages for all 9 modules with signatures, parameter semantics, return shapes, peer-dep notes, and examples, verified against the actual source exports. - Add the app-config row to the docs index so every link now resolves. - Package README, add a Usage section (ESM, CommonJS, framework notes), replace the stale 13-row table with grouped areas linking all 24 modules, correct the cog entry (pure helpers only after the migration), and drop em dashes.
…table - Remove 7 dead ./utils/* subpath rows from the Exports table. Those subpath exports were dropped in the utils-to-objex-utils move, so they 404 for consumers. Note that the pure utilities are re-exported from the package root and live in @walkthru-earth/objex-utils. - Add a Configuration section covering config.json, the ?config remote override, the ?panel/?rail/?tree query params, and the settings-panel precedence, plus a Configure bullet in Features.
TypeScript 5.9 to 6.0.3 across the root app and objex-utils. Added ignoreDeprecations "6.0" to the objex-utils tsconfig so tsup's DTS step does not fail on the baseUrl deprecation that TS 6 promotes to an error (baseUrl is injected by rollup-plugin-dts, not set in our config). Bumped to latest, verified against actual usage: - shiki 4, marked 18, pbf 5, @mapbox/vector-tile 3 - @babylonjs/core + @babylonjs/loaders 9, @carbonplan/zarr-layer 0.5 - @lucide/svelte 1, @types/node 25, @changesets/changelog-github 0.7 - dev: hyparquet 1.26, yaml 2.9 Breaking-change fixes: - pbf 5 dropped its default export, use PbfReader in pmtiles-tile.ts (@mapbox/vector-tile 3's VectorTile takes a PbfReader) - lucide 1 removed all brand icons, replaced the GitHub icon in AboutSheet with an inline SVG - @carbonplan/zarr-layer 0.5 targets zarrita 0.7 natively, so the pnpm patch and its file are removed - added babylonjs-gltf2interface 9.10.0 to satisfy babylon 9's peer Verified: objex-utils build plus 17 tests, svelte-check 0 errors, lint, and a production build all pass. vite 8 + plugin-svelte 7 and pnpm 11 were held back as higher-risk follow-ups.
Hide the panel-left toggle button when its target chrome is config-hidden: desktop requires showFileTree, mobile requires showConnectionRail or showFileTree (otherwise the sheet would open empty).
|
Important Review skippedToo many files! This PR contains 165 files, which is 15 over the limit of 150. To get a review, narrow the scope: ⚙️ Run configurationConfiguration used: defaults Review profile: CHILL Plan: Pro Run ID: ⛔ Files ignored due to path filters (1)
📒 Files selected for processing (165)
You can disable this status message by setting the Use the checkbox below for a quick retry:
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
The workspace symlink resolves @walkthru-earth/objex-utils to its built dist/, which does not exist after a fresh frozen install. Build the package immediately after install in both the CI and deploy workflows so type-checking and bundling can resolve every package import.
yharby
changed the title
The packages/**/*.ts glob matched the generated objex-utils dist/*.d.ts. This was latent until the workflow started building the package before format:check; the generated declarations are not formatted to our style and failed the check. Ignore **/dist/** and **/build/** so only source is formatted and linted.
The Svelte lib's .d.ts emit resolves types from @walkthru-earth/objex-utils, so the package must be built before `pnpm run package`. The previous order built it after, leaving the leading package build on the critical path with unresolved imports. Build objex-utils first, then package, then publish both.
Issue #11 happened because an unanticipated heavy dep leaked into the published bundle. The guardrail only hard-failed on an enumerated list; any dep not on the list was a passing warning, so a new unlisted heavy dep could regress the same way. objex-utils has zero runtime dependencies and four peer deps, so the only legitimate top-level bare imports are those peers plus relative/node:. Treat every other bare import as a hard failure (forbidden ones keep the clearer label). The enumerated FORBIDDEN_PATTERNS now only refine the error message, not the gate.
Merged
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Overview
This branch (
feat/utils-to-objex-utils) bundles 55 commits across five themes that landed together sincemain. Each theme has its own changeset, so the published@walkthru-earth/objexand@walkthru-earth/objex-utilspackages will bump together on merge.Files touched by area:
packages/objex-utils(37),src/lib/components(36),docs(23),src/lib/utils(10),src/lib/stores(6),src/lib/query(5),src/routes(3), plus storage, i18n, workflows, static config, and patches.1.
@walkthru-earth/objex-utilspackage split (utils-to-objex-utils,objex-utils-split-clarity)Relocated every framework-agnostic, pure-TypeScript utility out of
src/lib/utils/into the standalonepackages/objex-utils/src/package and deleted the old shim paths.wkb,hex,format,column-types,file-sort,error,lru,connection-identity,storage-smoketest,stac-storage-extension,cog-info,cog-asset,channel-composite,stac,stac-source,parquet-metadata,geoarrow,geometry-type,storage-url,cloud-url,host-detection,stac-facets,stac-pushdown,stac-geoparquet,markdown-sql. Consumers now import them from@walkthru-earth/objex-utils.src/lib/utils/keeps only heavy-dep or SvelteKit-bound modules. Renamedcog-puretocog-infoand the app-sideurltosigned-url.MarkdownSqlContext(host injects the engine).scripts/verify-objex-utils-bundle.mjs) that fails the build if any heavy dependency (@developmentseed/*,@deck.gl/*,zarrita,pdfjs-dist,shiki, etc.) leaks into the packagedist/, locking in the cog-info split lesson from Failed to resolve import "@developmentseed/epsg/all" from the pre-bundled objex-utils chunk. #11.2. Runtime config + settings panel (
runtime-config-settings-panel,config-basemaps-connections)Hosts can now customize objex without a rebuild.
static/config.json(or remote?config=<url>) seeds theme, locale, row and STAC item limits, chrome visibility, basemaps, and connection seeds. Malformed config falls back to safe defaults and still boots.AppConfig,mergeAppConfig,resolveSetting,parseVisibilityParam, value coercers) do a field-by-field merge of untrusted JSON with a first-match-wins precedence resolver. Unit tested.?config=,?panel=,?rail=,?tree=; toggles locked by a link param render read-only.MapContainerresolves its basemap through the new pureresolveBasemap()selector with a hardcoded CartoDB fallback. Connection rail seeds rows from config, auto-opening the first anonymous bucket and falling back to the Source Cooperative demo bucket.3. deck.gl-raster 0.7.0 upgrade (
upgrade-developmentseed-deckgl-raster-0-7-0)@developmentseed/deck.gl-{raster,geotiff,zarr,epsg,proj}from 0.6.1 to 0.7.0.RasterTileLayernatively, so the entiredeck.gl-rasterpnpm patch was removed.StacMosaicViewermigratedonTileUnloadto the new source-levelonSourceUnload.+overantimeridian fix, theinferRenderPipelinere-export, and ther16unormtor32floatFirefox/macOS texture fallback.4. Dependency upgrades + TypeScript 6
@carbonplan/zarr-layer, marked, pbf, shiki, lucide,@types/node, and others.PbfReader), lucide 1 (inline GitHub icon), dropped the orphaned zarr-layer patch.5. Storage, security, and CI hardening (
generic-s3-provider-and-url-fixes,redos-hardening)minioprovider to "MinIO / RustFS / Custom" for first-class S3-compatible store support (no id change). Fix "Copy HTTP URL" emitting an AWS URL for non-AWS connections via a shared provider-aware helper. Replace the misleading "Empty bucket" state with a CORS/network failure explanation.js/polynomial-redos) inconnection-identity,storage-url,cloud-url,markdown-sql, andcolumn-types. Same inputs, same outputs, linear worst case.objex-utilsimmediately after install so itsdist/exists before check, build, and bundle steps resolve@walkthru-earth/objex-utils(it resolves via the workspace symlink to built output).Test plan
pnpm run checkpasses (0 errors) once objex-utils is built firstpnpm run lintandpnpm run format:checkcleanvite buildsucceeds andpnpm run packagebuilds the lib tarball?rail=hide/?tree=hide