Skip to content

Chore/refactor hypersync#1809

Draft
PedroBinotto wants to merge 20 commits intodevfrom
chore/refactor-hypersync
Draft

Chore/refactor hypersync#1809
PedroBinotto wants to merge 20 commits intodevfrom
chore/refactor-hypersync

Conversation

@PedroBinotto
Copy link
Copy Markdown
Collaborator

No description provided.

@vercel
Copy link
Copy Markdown

vercel Bot commented Apr 8, 2026

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
anticapture Ready Ready Preview, Comment Apr 14, 2026 8:19pm
1 Skipped Deployment
Project Deployment Actions Updated (UTC)
anticapture-storybook Skipped Skipped Apr 14, 2026 8:19pm

Request Review

@claude
Copy link
Copy Markdown

claude Bot commented Apr 8, 2026

Claude encountered an error —— View job


I'll analyze this and get back to you.

@railway-app railway-app Bot temporarily deployed to anticapture-infra / dev April 8, 2026 16:39 Inactive
PedroBinotto and others added 2 commits April 10, 2026 14:44
HyperIndex uses PascalCase table names and camelCase column names.
The casing: "snake_case" option was overriding the explicit column
name strings in the schema, causing all table names to be lowercased
(e.g. "ProposalOnchain" -> "proposal_onchain").

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@claude
Copy link
Copy Markdown

claude Bot commented Apr 14, 2026

Claude encountered an error —— View job


I'll analyze this and get back to you.

Copy link
Copy Markdown

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 29c6f849f0

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

Comment on lines +123 to 124
"Transfer",
(drizzle) => ({
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Keep transaction aggregate SQL aligned with renamed tables

Renaming these tables/columns to quoted CamelCase breaks the raw SQL in apps/api/src/repositories/transactions/index.ts: getFilteredAggregateTransactions and filterToSql still reference legacy identifiers like transaction_hash, from_address, transfers.is_dex, and delegations.delegated_value. With this schema change, /transactions queries will hit undefined relation/column errors unless the SQL fragments are migrated to the new names (or compatibility views are added).

Useful? React with 👍 / 👎.

Comment on lines +148 to +152
export const votesOnchain = pgTable("VoteOnchain", (drizzle) => ({
id: drizzle.text().primaryKey(),
txHash: drizzle.text("txHash").notNull(),
daoId: drizzle.text("daoId").notNull(),
voterAccountId: drizzle.text("voterAccountId").$type<Address>().notNull(),
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Keep proposals activity SQL aligned with renamed on-chain tables

These renames are incompatible with apps/api/src/repositories/proposals-activity/index.ts, which still executes raw SQL against proposals_onchain/votes_onchain and snake_case columns (dao_id, tx_hash, voter_account_id, etc.) in getProposals, getUserVotes, and the paginated proposals query. After this commit, proposal-activity endpoints will fail against the new schema.

Useful? React with 👍 / 👎.

Comment on lines 200 to 202
export const daoMetricsDayBucket = pgTable(
"dao_metrics_day_buckets",
"DaoMetricsDayBucket",
(drizzle) => ({
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Keep DAO stats SQL aligned with renamed metrics/power tables

Changing this table naming/casing introduces runtime mismatches in apps/api/src/repositories/drizzle/index.ts, where comparison methods still query old identifiers such as dao_metrics_day_buckets, "account_power", "proposals_onchain", and "votes_onchain". Those dashboard/statistics queries will error once this schema is in place unless their raw SQL is updated to the new table/column names.

Useful? React with 👍 / 👎.

@vercel vercel Bot temporarily deployed to Preview – anticapture-storybook April 14, 2026 20:18 Inactive
@railway-app railway-app Bot temporarily deployed to anticapture-infra / dev April 14, 2026 20:18 Inactive
@pikonha pikonha marked this pull request as draft April 19, 2026 22:30
@pikonha
Copy link
Copy Markdown
Member

pikonha commented Apr 22, 2026

Code Review

Status: DRAFT · +5528 / −477 · labels: API, Indexer

Overview

This PR introduces a new Envio/HyperIndex indexer (@anticapture/hypersync-indexer) as a replacement for the Ponder-based indexer, and migrates the API's Drizzle schema to match Envio's default naming convention. Changes span three areas:

  1. New app apps/hypersync-indexer/ — Envio project for ENS token + governor with handlers for transfers, delegations, voting, and supply metrics, plus a root Dockerfile.hypersync-indexer.
  2. API schema rewrite (apps/api/src/database/schema.ts) — tables renamed snake_casePascalCase (account_balanceAccountBalance), columns renamed to camelCase, casing: \"snake_case\" dropped from Drizzle clients, composite primary keys replaced with synthetic id text PKs, DATABASE_URL no longer injects search_path=anticapture.
  3. Test sweep — every unit/integration test updated for the new schema shape.

Must-fix

  • Empty ts-node file at repo root (diff --git a/ts-node b/ts-node, 0 bytes). Almost certainly an accidental artifact of npm install with a typo / redirected output. Delete.
  • Empty PR description on a 5.5k-line structural migration. Please add: motivation, rollout/migration plan for prod DBs, coexistence story with the existing Ponder indexer, rollback procedure. DRAFT is fine, but reviewers need this.
  • Lost DB-level uniqueness constraints. The old schema had composite PKs: AccountBalance(accountId, tokenId), AccountPower(accountId), VotingPowerHistory(transactionHash, accountId, logIndex), BalanceHistory(...). They're now synthetic id text primary key columns with no unique index on the natural key. Integrity is now guaranteed only by the indexer's id-construction convention (\${accountId}-\${tokenId}, etc.). Add unique indexes on the natural keys — this is the kind of thing you want enforced by Postgres, not application code.
  • DATABASE_URL search_path override removed (apps/api/src/env.ts) and casing: \"snake_case\" dropped (cmd/index.ts, cmd/aave.ts). Confirm Envio writes to whatever schema the API reads from (previously the anticapture schema was forced). If Envio writes to public by default, the API now silently reads from public too. Document the expected DB layout and whether existing anticapture-schema data is being dropped.
  • apps/hypersync-indexer/generated/package.json is checked in but the surrounding .gitignore ignores /generated/* with an exception for this one file. envio codegen regenerates the whole directory; pinning package.json will produce diff churn and drift vs. the installed Envio version. Either gitignore the whole folder and have CI run envio codegen && pnpm install in installer, or commit the full generated/ (not recommended). The current hybrid is the worst option.

Should-fix

  • Dockerfile.hypersync-indexer
    • envio is in devDependencies, but the runner sets NODE_ENV=production and the entrypoint runs npm run startenvio start. If pnpm install ever skips devDeps, the binary disappears. Move envio to dependencies or set --prod=false explicitly.
    • npm install --prefix generated --legacy-peer-deps in a pnpm monorepo undermines lockfile determinism — this is why committing generated/package.json is fragile. Prefer pnpm install at the generated dir, or regenerate at build time and install once.
    • Installs curl and ts-node globally in both installer and runner stages — collapse into the stage that needs it.
    • No USER directive (runs as root). No HEALTHCHECK. Standard for a production indexer.
  • handleTransaction early-returns when no flag matches (shared.ts). Correct for intent (only persisting flagged txs), but the function name suggests unconditional persistence. Rename or add a one-line comment.
  • storeDailyBucket.average uses integer bigint division (shared.ts), which will drift for large counts. Same pattern as before, so not a regression, but worth noting.
  • Enum string mapping duplicated between MetricTypesEnum (TS) and metricType: MetricType_t (generated from schema.graphql). Any drift is silent. Consider generating one from the other or adding an exhaustive-check.
  • _delegatorBalance leading underscore (delegation.ts) — the variable is read immediately after, so the underscore is misleading. Drop it.
  • ensureAccountExists + ensureAccountsExist duplicate each other; keep only the batched version.
  • delegation.ts delegateChanged accumulates existingDelegation.delegatedValue + delegatedValue. DelegateChanged per delegator-per-tx is normally unique, but if the id ever collides (re-org replay, same tx replayed), this double-counts. Worth a comment on intent.
  • No handler tests: the new indexer has ~1.5k LOC of business logic and zero tests. At minimum, unit tests for handleTransaction, storeDailyBucket, and delegateChanged would catch future regressions. The PR updates API tests exhaustively — please extend that rigor to the new package.

Nits

  • eslint.config.mjs — the comment says "const + type with same name pattern"; point to the file in src/lib/ where that pattern lives so future readers don't re-litigate.
  • apps/hypersync-indexer/tsconfig.json includes generated/**/*.ts with allowImportingTsExtensions: true + noEmit — only meaningful at typecheck time. Worth documenting for newcomers.
  • Root package.json adds hypersync-indexer turbo script — consistent with existing indexer/api commands, good.

Verification checklist

  • Start Envio locally against a fresh Postgres; confirm API reads the same schema Envio writes (search_path alignment).
  • Run pnpm api typecheck && pnpm api test after schema cutover.
  • Spot-check that row.Transfer / row.Delegation join aliases in voting-power/general.ts and nouns.ts actually match Drizzle's relation inference for the new PascalCase table names — this typechecks but could fail at runtime.
  • Confirm rollback plan: can we re-run the old Ponder indexer against the same DB if the Envio cutover fails? If not, note that explicitly.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants