Skip to content

improve: tighten ingestion and SDK types#11

Merged
remcostoeten merged 1 commit into
masterfrom
feature/type-safe-ingestion-sdk-cleanup
Apr 22, 2026
Merged

improve: tighten ingestion and SDK types#11
remcostoeten merged 1 commit into
masterfrom
feature/type-safe-ingestion-sdk-cleanup

Conversation

@remcostoeten
Copy link
Copy Markdown
Owner

@remcostoeten remcostoeten commented Apr 22, 2026

Summary

Tightens the analytics ingestion and SDK type surface while cleaning up legacy schema usage.

Changes

  • Fixes ORIGIN_ALLOWLIST so configured allowlists actually reject unlisted origins.
  • Removes active Drizzle schema exports/registration for legacy resume and visitor_events tables.
  • Adds a non-destructive migration that archives those legacy tables as old_resume and old_visitor_events.
  • Improves SDK public types for LSP support, including exported AnalyticsProps, JSON-safe TrackMeta, and custom string event names.
  • Removes runtime any usage from ingestion and SDK source paths.
  • Preserves "use client" in built SDK outputs with a post-build helper.
  • Excludes generated dist output from root formatting checks.

Validation

  • bun run typecheck
  • bun run test
  • bun run lint
  • bun run fmt:check
  • bun run build in packages/sdk

DB Migration

Migration file added at apps/ingestion/src/db/migrations/0001_archive_legacy_tables.sql. It renames existing legacy tables only if present and does not drop data.

Summary by Sourcery

Tighten ingestion and SDK typing while removing legacy database usage and preserving client runtime behavior in the built SDK.

New Features:

  • Expose richer, JSON-safe SDK typing via TrackMeta, AnalyticsProps, and extensible event name support.
  • Add a post-build script to inject the "use client" directive into built SDK entry files.

Bug Fixes:

  • Ensure ORIGIN_ALLOWLIST correctly blocks requests from non-allowlisted origins instead of allowing them.
  • Strengthen ingestion seeding and SDK utilities to avoid unsafe any usage and handle missing env/connection properties more robustly.

Enhancements:

  • Remove legacy resume and visitor_events table schema exports and their usage from the ingestion service.
  • Refine database client creation with a typed fallback client when DATABASE_URL is absent.
  • Tighten types across the SDK (tracking APIs, enrichment, performance observers) for better LSP and type safety.
  • Update ingestion middleware and seeding scripts to use stricter typing and cleaner logging output.

Build:

  • Adjust root formatting commands to ignore generated dist outputs in apps and packages.
  • Update the SDK tsconfig include paths to cover build scripts used in the packaging pipeline.
  • Change tsup configuration to run a post-build script instead of using an esbuild banner for client directives.

Tests:

  • Add an ingestion route test verifying that requests from origins outside the configured ORIGIN_ALLOWLIST are rejected.
  • Update schema tests to focus on current visitors schema instead of removed legacy tables.

Chores:

  • Introduce a non-destructive migration that archives legacy resume and visitor_events tables to old_resume and old_visitor_events if present.

Summary by CodeRabbit

  • Bug Fixes

    • Origin allowlist validation now properly enforces origin restrictions for ingest requests.
  • Refactor

    • Archived legacy database tables and updated schema exports.
    • Enhanced type safety across SDK tracking functions and database interactions.
    • Improved build configuration to exclude generated output from formatting checks.

@vercel
Copy link
Copy Markdown

vercel Bot commented Apr 22, 2026

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
analytics-demo Ready Ready Preview, Comment Apr 22, 2026 5:24pm
ingestion Ready Ready Preview, Comment Apr 22, 2026 5:24pm

@coderabbitai
Copy link
Copy Markdown

coderabbitai Bot commented Apr 22, 2026

Caution

Review failed

Pull request was closed or merged during review

📝 Walkthrough

Walkthrough

The PR removes legacy database tables (resume, visitorEvents) from the ingestion schema and client, refactors DB client construction with explicit typing, enforces origin-based request filtering for the ingest endpoint, and improves type safety across the SDK by introducing TrackMeta for event metadata, shifting the "use client" directive from esbuild banner to post-build script, and strengthening typed constraints in enrichment and observer utilities.

Changes

Cohort / File(s) Summary
Ingestion: Database Schema & Migration
apps/ingestion/src/db/schema.ts, apps/ingestion/src/db/migrations/0001_archive_legacy_tables.sql
Removed resume and visitorEvents table definitions and their inferred types; added SQL migration to rename legacy tables to old_resume and old_visitor_events.
Ingestion: Database Client & Configuration
apps/ingestion/src/db/client.ts, apps/ingestion/src/db/index.ts
Refactored DB client construction with createDb() factory and DbClient type alias; narrowed Drizzle schema to { events, visitors }; introduced createFallbackDb() for offline fallback; updated exports to remove resume and visitorEvents.
Ingestion: Seed & Schema Validation
apps/ingestion/src/db/seed.ts, apps/ingestion/tests/unit/schema.test.ts
Strengthened seed typing with NewEvent[] and SeedMeta; added getRegions()/getCities() typed helpers; removed emoji logging; removed legacy table assertions from schema tests.
Ingestion: Request Handling & Origin Allowlisting
apps/ingestion/src/handlers/ingest.ts, apps/ingestion/tests/unit/ingest.test.ts
Added runtime origin allowlist computation via getOriginAllowlist() and enforced blocking of disallowed origins with 403 response; typed __setDbModule() parameter; added test case for rejected origin.
Ingestion: Middleware Type Safety
apps/ingestion/src/app.ts
Added MiddlewareHandler import and constrained requestCounter() return type via satisfies for improved middleware typing.
SDK: Core Type System & Enrichment
packages/sdk/src/types/index.ts, packages/sdk/src/utilities/enrichment.ts
Introduced JsonPrimitive/JsonValue/TrackMeta for JSON-shaped metadata; broadened EventType to include string literals; added AnalyticsProps type; made EventPayload generic; typed network enrichment helpers to avoid any casts.
SDK: Track API & Observer Typing
packages/sdk/src/api/track.ts, packages/sdk/src/observers/performance.ts
Updated track API functions to accept meta?: TrackMeta; refined getEnv() with ImportMetaEnv typing; added LayoutShift/InteractionEntry aliases for CLS and INP observers.
SDK: Build & Directive Injection
packages/sdk/scripts/client-directive.ts, packages/sdk/tsup.config.ts
Introduced post-build script to prepend "use client"; to bundled output files; replaced esbuild banner injection with onSuccess command.
SDK: Component & Public API
packages/sdk/src/components/analytics.tsx, packages/sdk/src/index.ts
Removed "use client"; directives from component and index; updated Analytics prop type to AnalyticsProps; maintained all exports and public signatures.
SDK & Root Configuration
packages/sdk/tsconfig.json, package.json
Extended SDK TypeScript includes to cover scripts/**/*.ts; updated formatting scripts to exclude dist/** directories.

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~25 minutes

Possibly related PRs

Poem

🐰 Tidy tables tucked away, schemas clean in every way,
Types now flow like morning dew, "use client" knows what's true.
Legacy gone, new safety found, metadata in types so sound!

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The PR title 'improve: tighten ingestion and SDK types' accurately captures the primary objective of the changeset, which focuses on enhancing type safety across the ingestion and SDK modules.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch feature/type-safe-ingestion-sdk-cleanup

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@sourcery-ai
Copy link
Copy Markdown

sourcery-ai Bot commented Apr 22, 2026

Reviewer's Guide

Tightens ingestion and SDK typing, removes legacy resume/visitor_events tables from the active schema with a non-destructive archival migration, fixes ORIGIN_ALLOWLIST enforcement, and adds build/runtime safeguards including a safer DB client fallback and preserving "use client" in built SDK outputs.

Sequence diagram for ingestion request with origin allowlist and DB fallback

sequenceDiagram
  actor Browser
  participant SDK as Analytics_SDK
  participant Ingestion as Ingestion_handler
  participant Env as Process_env
  participant DbFactory as Db_client_factory
  participant Postgres

  Browser->>SDK: track(...)
  SDK->>Ingestion: POST /ingest (EventPayload)
  Ingestion->>Env: read ORIGIN_ALLOWLIST
  Env-->>Ingestion: comma_separated_origins
  Ingestion->>Ingestion: getOriginAllowlist()
  Ingestion->>Ingestion: isOriginAllowed(request_origin)
  alt origin allowed or no allowlist configured
    Ingestion->>DbFactory: getDbClient()
    DbFactory->>Env: read DATABASE_URL
    alt DATABASE_URL present
      Env-->>DbFactory: database_url
      DbFactory->>Postgres: connect via neon
      DbFactory-->>Ingestion: db_client
    else DATABASE_URL missing
      Env-->>DbFactory: undefined
      DbFactory-->>Ingestion: fallback_db_client
    end
    Ingestion->>Postgres: insert into events
    Postgres-->>Ingestion: insert result
    Ingestion-->>SDK: 200 OK
  else origin not in allowlist
    Ingestion-->>SDK: 403 Forbidden
  end
Loading

ER diagram for updated analytics ingestion schema and archived legacy tables

erDiagram
  visitors {
    bigint id PK
    text visitor_id
    text project_id
    timestamp created_at
  }

  events {
    bigint id PK
    text project_id
    text type
    timestamp ts
    jsonb meta
  }

  old_resume {
    bigint id PK
    text event
    timestamp ts
    text path
    jsonb meta
  }

  old_visitor_events {
    bigint id PK
    bigint visitor_id
    text event_type
    timestamp ts
    jsonb meta
  }

  visitors ||--o{ events : has_many

  visitors ||--o{ old_visitor_events : had_many
Loading

Class diagram for updated SDK analytics types and tracking API

classDiagram
  class JsonPrimitive {
  }

  class JsonValue {
  }

  class TrackMeta {
  }

  class KnownEventType {
  }

  class EventType {
  }

  class AnalyticsOptions {
    projectId string
    ingestUrl string
    debug boolean
  }

  class AnalyticsProps {
    projectId string
    ingestUrl string
    debug boolean
    disabled boolean
  }

  class EventPayload {
    type EventType
    projectId string
    path string
    referrer string
    userAgent string
    screen string
    viewport string
    lang string
    visitorId string
    sessionId string
    meta TrackMeta
  }

  class AnalyticsComponent {
    +Analytics(projectId string, ingestUrl string, disabled boolean, debug boolean)
  }

  class TrackApi {
    +track(type EventType, meta TrackMeta, options AnalyticsOptions) void
    +trackPageView(meta TrackMeta, options AnalyticsOptions) void
    +trackEvent(eventName string, meta TrackMeta, options AnalyticsOptions) void
    +trackClick(elementName string, meta TrackMeta, options AnalyticsOptions) void
    +trackError(error Error, meta TrackMeta, options AnalyticsOptions) void
  }

  JsonPrimitive <|-- JsonValue
  JsonValue <|-- TrackMeta
  KnownEventType <|-- EventType
  AnalyticsOptions <|-- AnalyticsProps
  AnalyticsOptions <.. TrackApi
  EventPayload o-- TrackMeta
  TrackApi --> EventPayload
  AnalyticsComponent --> AnalyticsProps
Loading

File-Level Changes

Change Details Files
Remove legacy resume and visitor_events tables from active Drizzle schema and archive them via SQL migration.
  • Delete resume and visitorEvents table definitions and related inferred types from the ingestion schema module.
  • Stop exporting legacy tables and types from the DB index and Drizzle client schema registration.
  • Add a SQL migration that conditionally renames resume and visitor_events tables to old_resume and old_visitor_events without dropping data.
  • Update schema unit tests to only assert events/visitors schemas and drop expectations around legacy tables.
apps/ingestion/src/db/schema.ts
apps/ingestion/src/db/index.ts
apps/ingestion/src/db/client.ts
apps/ingestion/tests/unit/schema.test.ts
apps/ingestion/src/db/migrations/0001_archive_legacy_tables.sql
Strengthen typing across ingestion seed data, DB client, middleware, SDK tracking API, enrichment utilities, and performance observers to eliminate runtime any usage and improve safety.
  • Make the seed script strongly typed using NewEvent and JSON-safe SeedMeta/SeedValue types, plus typed helper functions for regions/cities.
  • Replace loosely typed DB client factory with createDb/createFallbackDb helpers and a concrete DbClient type rather than any.
  • Tighten __setDbModule and middleware types in ingestion handler, and add typed origin allowlist helper.
  • Introduce JsonValue and TrackMeta types and use them across the SDK (track API, enrichment utilities, event payload meta), and export AnalyticsProps for React usage.
  • Add typed LayoutShift/InteractionEntry wrappers for performance observers and a typed NavigatorConnection for network info enrichment.
apps/ingestion/src/db/seed.ts
apps/ingestion/src/db/client.ts
apps/ingestion/src/handlers/ingest.ts
packages/sdk/src/types/index.ts
packages/sdk/src/api/track.ts
packages/sdk/src/components/analytics.tsx
packages/sdk/src/observers/performance.ts
packages/sdk/src/utilities/enrichment.ts
Fix ORIGIN_ALLOWLIST so only configured origins are allowed and add test coverage.
  • Replace static ORIGIN_ALLOWLIST array with a getOriginAllowlist helper that parses and trims the env var each time.
  • Update isOriginAllowed to return false when an origin is not in the allowlist and the list is non-empty instead of always allowing.
  • Add a unit test that sets ORIGIN_ALLOWLIST, sends a request with a non-allowlisted Origin header, and asserts a 403 with an appropriate error payload.
apps/ingestion/src/handlers/ingest.ts
apps/ingestion/tests/unit/ingest.test.ts
Improve SDK public API surface and runtime behavior, including JSON-safe meta, custom event names, and preserved "use client" directive in builds.
  • Expand EventType to allow branded custom strings while retaining known core event types.
  • Make EventPayload generic over EventType and switch meta to TrackMeta for JSON-safe metadata shapes.
  • Replace local Analytics props type with exported AnalyticsProps and update the Analytics component to use it.
  • Ensure built SDK bundles include the "use client" directive via an onSuccess tsup hook that runs a post-build script to prepend the directive to dist entry files.
  • Remove top-level "use client" and banner-based injection from SDK entry/tsup config in favor of the post-build script.
packages/sdk/src/types/index.ts
packages/sdk/src/components/analytics.tsx
packages/sdk/src/api/track.ts
packages/sdk/src/index.ts
packages/sdk/tsup.config.ts
packages/sdk/scripts/client-directive.ts
Adjust tooling and formatting to accommodate generated outputs and build scripts.
  • Exclude dist/** from root formatting and fmt:check commands so generated bundles are not reformatted.
  • Extend the SDK tsconfig include paths to cover scripts/**/*.ts so the client-directive build script is typechecked.
package.json
packages/sdk/tsconfig.json

Tips and commands

Interacting with Sourcery

  • Trigger a new review: Comment @sourcery-ai review on the pull request.
  • Continue discussions: Reply directly to Sourcery's review comments.
  • Generate a GitHub issue from a review comment: Ask Sourcery to create an
    issue from a review comment by replying to it. You can also reply to a
    review comment with @sourcery-ai issue to create an issue from it.
  • Generate a pull request title: Write @sourcery-ai anywhere in the pull
    request title to generate a title at any time. You can also comment
    @sourcery-ai title on the pull request to (re-)generate the title at any time.
  • Generate a pull request summary: Write @sourcery-ai summary anywhere in
    the pull request body to generate a PR summary at any time exactly where you
    want it. You can also comment @sourcery-ai summary on the pull request to
    (re-)generate the summary at any time.
  • Generate reviewer's guide: Comment @sourcery-ai guide on the pull
    request to (re-)generate the reviewer's guide at any time.
  • Resolve all Sourcery comments: Comment @sourcery-ai resolve on the
    pull request to resolve all Sourcery comments. Useful if you've already
    addressed all the comments and don't want to see them anymore.
  • Dismiss all Sourcery reviews: Comment @sourcery-ai dismiss on the pull
    request to dismiss all existing Sourcery reviews. Especially useful if you
    want to start fresh with a new review - don't forget to comment
    @sourcery-ai review to trigger a new review!

Customizing Your Experience

Access your dashboard to:

  • Enable or disable review features such as the Sourcery-generated pull request
    summary, the reviewer's guide, and others.
  • Change the review language.
  • Add, remove or edit custom review instructions.
  • Adjust other review settings.

Getting Help

@remcostoeten remcostoeten changed the title [codex] tighten ingestion and SDK types improve: tighten ingestion and SDK types Apr 22, 2026
@remcostoeten remcostoeten marked this pull request as ready for review April 22, 2026 17:28
@remcostoeten remcostoeten merged commit 0af6531 into master Apr 22, 2026
6 of 7 checks passed
Copy link
Copy Markdown

@sourcery-ai sourcery-ai Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey - I've found 2 issues, and left some high level feedback:

  • The new client-directive.ts script relies on import.meta.dirname, which is still relatively environment-specific; consider resolving the dist directory via fileURLToPath(new URL("../dist", import.meta.url)) to avoid breakage if the Node/runtime support for import.meta.dirname changes.
  • The createFallbackDb implementation silently turns all DB operations into no-ops when DATABASE_URL is missing; it may be safer to at least log a clear warning or throw outside build/test contexts so configuration errors don’t get masked at runtime.
Prompt for AI Agents
Please address the comments from this code review:

## Overall Comments
- The new `client-directive.ts` script relies on `import.meta.dirname`, which is still relatively environment-specific; consider resolving the dist directory via `fileURLToPath(new URL("../dist", import.meta.url))` to avoid breakage if the Node/runtime support for `import.meta.dirname` changes.
- The `createFallbackDb` implementation silently turns all DB operations into no-ops when `DATABASE_URL` is missing; it may be safer to at least log a clear warning or throw outside build/test contexts so configuration errors don’t get masked at runtime.

## Individual Comments

### Comment 1
<location path="packages/sdk/scripts/client-directive.ts" line_range="8" />
<code_context>
+const directive = '"use client";';
+
+for (const file of files) {
+	const path = join(import.meta.dirname, "..", "dist", file);
+	const source = readFileSync(path, "utf8");
+	if (source.startsWith(directive)) continue;
</code_context>
<issue_to_address>
**issue:** Using `import.meta.dirname` is non-standard and will likely cause type/runtime issues.

`import.meta.dirname` isn’t part of the standard `ImportMeta` and isn’t supported by Node’s ESM runtime (which only provides `import.meta.url`), so TypeScript will flag it and the script becomes Bun-specific.

To keep this portable and type-safe, you can derive the directory from `import.meta.url` instead:

```ts
import { fileURLToPath } from "node:url";

const __dirname = fileURLToPath(new URL("..", import.meta.url));
const path = join(__dirname, "dist", file);
```

This stays within standard ESM semantics and avoids custom `ImportMeta` typing.
</issue_to_address>

### Comment 2
<location path="apps/ingestion/src/db/client.ts" line_range="12-21" />
<code_context>
+
+type DbClient = ReturnType<typeof createDb>;
+
+function createFallbackDb(): DbClient {
+	return {
+		select() {
+			return {
+				from() {
+					return [];
+				},
+			};
+		},
+		insert() {
+			return {
+				values() {
+					return {
+						returning() {
+							return [];
+						},
+					};
+				},
+			};
+		},
+		async execute() {
+			return { rows: [] };
+		},
+	} as unknown as DbClient;
+}
+
</code_context>
<issue_to_address>
**suggestion (bug_risk):** Fallback DB client shape diverges from the real drizzle client and may cause subtle runtime differences.

The fallback client is cast to `DbClient`, but its query methods are synchronous and return plain arrays/objects, while the real drizzle client returns Promises (e.g. `select().from()`, `insert().values().returning()`). This can break code that uses Promise-specific behavior (`.then`, `Promise.all`, assuming async `execute()`). Consider making these methods `async` and returning Promise-based results that mirror drizzle’s shapes to keep the async contract consistent.
</issue_to_address>

Sourcery is free for open source - if you like our reviews please consider sharing them ✨
Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.

const directive = '"use client";';

for (const file of files) {
const path = join(import.meta.dirname, "..", "dist", file);
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

issue: Using import.meta.dirname is non-standard and will likely cause type/runtime issues.

import.meta.dirname isn’t part of the standard ImportMeta and isn’t supported by Node’s ESM runtime (which only provides import.meta.url), so TypeScript will flag it and the script becomes Bun-specific.

To keep this portable and type-safe, you can derive the directory from import.meta.url instead:

import { fileURLToPath } from "node:url";

const __dirname = fileURLToPath(new URL("..", import.meta.url));
const path = join(__dirname, "dist", file);

This stays within standard ESM semantics and avoids custom ImportMeta typing.

Comment on lines +12 to +21
function createFallbackDb(): DbClient {
return {
select() {
return {
from() {
return [];
},
};
},
insert() {
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

suggestion (bug_risk): Fallback DB client shape diverges from the real drizzle client and may cause subtle runtime differences.

The fallback client is cast to DbClient, but its query methods are synchronous and return plain arrays/objects, while the real drizzle client returns Promises (e.g. select().from(), insert().values().returning()). This can break code that uses Promise-specific behavior (.then, Promise.all, assuming async execute()). Consider making these methods async and returning Promise-based results that mirror drizzle’s shapes to keep the async contract consistent.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant