hk-open-data

Effect-based TypeScript tools for discovering, synchronizing, inspecting, and retrieving English DATA.GOV.HK catalog resources.

Status: pre-release at 0.0.0. The first npm release will be 0.1.0 only when it is deliberately approved.

What is public

This repository contains one future npm package, hk-open-data:

• a reusable core library for CKAN and DATA.GOV.HK APIs;
• a Bun CLI (hk-open-data / hk-data);
• a portable Effect synchronization engine;
• a local SQLite synchronization and catalog adapter;
• provenance-aware retrieval contracts and agent skill guidance.

Hosted services and their deployment configuration live in a separate private repository. They consume this package rather than changing its public core.

Install

The package has not been published yet. From a source checkout:

bun install --frozen-lockfile
bun run typecheck
bun test

After the first release:

npm install hk-open-data
npx hk-open-data --help

The package manager for consumers is npm. Bun remains the runtime because local catalog access uses bun:sqlite.

Synchronize a local catalog

bun run sync -- --db data/catalog.sqlite

sync skips a catalog completed within the last 24 hours. Use --force to refresh immediately, or --max-age-hours <hours> to choose a different threshold. For a bounded smoke test, use --limit 1 --force with a temporary --db; partial runs never mark the main catalog fresh or prune unseen packages.

The sync engine is storage-neutral. The public repository supplies the local SQLite store; a private Cloudflare service can provide a D1 store and invoke the same Effect program from a Cron trigger.

Use the CLI

bun run --filter @hk-open-data/cli dev -- search-packages "traffic" --has-api
bun run --filter @hk-open-data/cli dev -- search-resources "arrival" --kind api

Set HK_OPEN_DATA_CATALOG_DB or pass --db to select the SQLite catalog.

Versioning and releases

The project uses Semantic Versioning and Changesets. While the package is in 0.x, patch versions are compatible fixes and minor versions may contain features or breaking changes.

Changesets collect release intent, but the release workflow remains disabled until the repository variable NPM_RELEASE_ENABLED is explicitly set to true. Publishing is then gated by the release pull request, the protected main branch, the GitHub npm environment, and npm trusted publishing. No npm token is stored in the repository.

Repository layout

packages/core/          shared Effect services, catalog contracts, and TypeScript sync
packages/cli/           thin command-line adapter
packages/hk-open-data/  the single future npm package
packages/skills/        public agent guidance
scripts/                OpenAPI utilities and Python-only LiteParse document workflows
specs/openapi/          imported and normalized platform specifications
docs/                   design notes, ADRs, source notes, and harvest findings

See CONTRIBUTING.md, SECURITY.md, and LICENSE. Code is MIT licensed; source datasets retain their original terms and attribution requirements.