Skip to content

feat(redis): keys as tables — lists, sets, sorted sets, streams and JSON documents - #552

Merged
huyplb merged 5 commits into
mainfrom
feat/redis-tables
Oct 25, 2026
Merged

huyplb merged 5 commits into
mainfrom
feat/redis-tables

Conversation

@huyplb

@huyplb huyplb commented Oct 25, 2026 •

Copy link
Copy Markdown
Contributor

What this adds

Redis in the SQL editor reads like the rest of the app.

  • The schema explorer lists the keyspace. Before, it showed "No provider registered for dialect: redis" and listed nothing. Now there is a schema provider: a prefix (users for users:*) is a table, and a list, set, sorted set or stream key is a table of its own. Each comes with a type badge. The listing is from a bounded SCAN sample and says when it is partial.
  • Every key type reads as rows.
    • A list, set, sorted set or stream key reads as its elements. Before, it read as just { id, type }.
    • A JSON document (the JSON type, or JSON text in a string) reads as its top-level fields.
    • JSON elements in a list or stream widen the table.
    • Reads are bounded: a page reads only what it shows, and a WHERE stops after 50,000 elements.
  • Paging works. The editor wrapped every SELECT in SELECT * FROM (…) LIMIT …, which the Redis translator refused with "Unsupported clause". The page now goes onto the statement itself (pageSubsetSelect). The same fix applies to MongoDB.
  • A passwordless Redis or MongoDB is usable. It is asked for a password only when the connection names a user. Every gate reads one rule, needsSessionPassword.
  • Refusals. Writes to a collection or a JSON document are refused with a reason. Force-migrate refuses a target with no SQL DDL: for a Redis target it used to fall back to Db2 DDL. The explorer offers no table to create and no blueprint for Redis.

Simplify and review pass (2026-10-25)

Four reviewers looked at reuse, simplification, efficiency and altitude.

  • Keyspace reader:
    • One table describes the four collection types.
    • Reads are pipelined: a 40-prefix sample went from about 4,000 sequential round trips to a few.
    • There is one TYPE per key, and UPDATE asks TYPE once.
  • The subset parser reuses splitTopLevel / splitTop. isSubsetDialect is derived from the registry instead of a hand-kept list.
  • The password rule is now one helper where there were 12 copies. A test that mirrored a stale copy now calls the real one.
  • e2e:
    • helpers/redis.ts seeds through one redis-cli --pipe per database.
    • redis-browse is folded into redis-editor.
    • openDataPeek can wait for rows on a read-only peek.

Tests

  • Unit: the keyspace reader against an in-memory node-redis double, plus the adapter, provider, subset parser, password rule and force-migrate guard. The new efficiency and guard tests fail on the old code and pass on the new.
  • Live: redis.it.test.ts passes 19/19 against the dev container (node-redis 6.3.0).
  • e2e: redis-editor.test.ts passes 21/21. It covers basic SQL, every built-in sample, Compare data with Data migrate, the schema explorer, and Data peek with the row editor.
  • Gates: typecheck (web, e2e, cli), eslint . and the security lint are clean. Unit suite: 6517 pass. The 2 failures are the known timing flakes (code-cell-execute, migration-file linear time); neither file is touched here.

Not in this PR

  • A Redis with only requirepass, saved without its password, is not prompted. The rule keys on the username, so that password stays saved on the connection. An explicit "No password" sign-in choice is on the board.
  • The workflow SQL step's password warning ignores this rule. Its connection summary has no username. Filed separately.

🤖 Generated with Claude Code


Note

Medium Risk
Touches SQL execution, connection auth gating, and a large Redis adapter/keyspace surface; behavior changes for Mongo paging and password prompts, but guarded by extensive unit/integration/e2e tests.

Overview
Redis (and MongoDB subset paging) in the SQL editor — keyspaces appear in the schema explorer with store-type badges; lists, sets, sorted sets, streams, and JSON documents render as queryable rows, not stub { id, type } rows. A new Redis schema provider and redis.keyspace layer power browsing and SQL execution; hash-only writes stay enforced with explicit errors for collections and JSON.

Paging and errors — subset dialects no longer use a nested SELECT * FROM (…) AS fox_page; the server appends LIMIT/OFFSET via pageSubsetSelect, and Redis/Mongo statement failures surface without a misleading “Paging failed” wrapper. MongoDB OFFSET paging and empty limit(0) pages are fixed similarly.

Auth UX — needsSessionPassword centralizes when the UI must prompt (passwordless Redis/Mongo when no username; Redis with ACL user still needs a secret). Compare retest gates and editor destination selection use this helper.

Explorer / DDL guardrails — New table / blueprint actions hide for engines without SQL DDL (Redis, MongoDB). Subset explorers insert bare table names and column refs without aliases.

History & migrate — Schema capture, force-migrate, and related routes refuse Redis/Mongo before loading a sampled keyspace as a version or emitting wrong DDL.

E2E — redis-editor.test.ts (docker redis-cli helper) exercises SQL, samples, compare/migrate, explorer, and data peek; test IDs on grid/run/result errors support assertions.

Parser hardening — SQL subset gains quoted dotted table names, parenthesized WHERE (Data peek), stricter predicate/column rules, and semicolon-safe autoAliasSelectColumns.

Reviewed by Cursor Bugbot for commit 90416f7. Bugbot is set up for automated code reviews on this repo. Configure here.

huyplb and others added 5 commits October 25, 2026 09:33
# Conflicts:
#	apps/e2e/scripts/run-all.mjs
#	apps/web/src/frontend/features/sql-editor/components/ResultsPanel.tsx
…rule

From the simplify review of this branch:

- Keyspace reader: the four collection types are one table (columns,
  length, chunks, indexed), so a new type is one entry. Reads are sized to
  the page (LRANGE/XRANGE no longer fetch 500 to show 20), sampling reads
  every prefix's keys together with the types already known (one TYPE per
  key), and the prefix SELECT reads a SCAN batch at once through takePage,
  which also bounds a WHERE that matches nothing at MAX_SCAN keys.
- readKey returns the key's type, so UPDATE asks TYPE once.
- needsSessionPassword: the one copy of "ask for a password first",
  replacing 12 hand-written copies (and a test that mirrored a stale one).
- isSubsetDialect derives from the registry (a known engine with no SQL
  dialect) instead of a hand-kept list; the subset parser reuses
  splitTopLevel and splitTop instead of two more scanners.
- Force-migrate refuses a target without SQL DDL: a Redis target reached
  resolveDialect, which answers Db2.
- The explorer decides "can create a table" itself and hides all three
  create entry points and the blueprint for Redis, not just the header.
- e2e: helpers/redis.ts seeds through one redis-cli --pipe per database;
  redis-browse folded into redis-editor; openDataPeek can wait for rows.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… time

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@cursor

cursor Bot commented Oct 25, 2026

Copy link
Copy Markdown
Contributor

Bugbot couldn't run - usage limit reached

Bugbot is counted against Cursor usage for this user or team, and this run hit a usage or spend limit.

A user or team admin can review and increase usage limits in the Cursor dashboard.

(requestId: b675247b-b927-43be-b009-7a1384114700)

@github-actions

Copy link
Copy Markdown
Contributor

Test IDs

Added (4)

  • sql-explorer-store-type-{…}
  • sql-grid-error
  • sql-result-error
  • sql-run-error

{…} is a part filled in at run time. The full tree: docs/testing/TEST_IDS.md.

import { useSqlEditorStore } from '@/features/sql-editor/state';
import { useAuthStore } from '@/platform/authStore';
import { PROVIDER_SETTINGS, connectionNeedsSecret } from '@/shared/lib/provider-settings';
import { PROVIDER_SETTINGS, needsSessionPassword } from '@/shared/lib/provider-settings';
import { useSyncStore } from '@/features/compare';
import { useSqlEditorStore } from '@/features/sql-editor/state';
import { PROVIDER_SETTINGS, connectionNeedsSecret } from '@/shared/lib/provider-settings';
import { PROVIDER_SETTINGS, needsSessionPassword } from '@/shared/lib/provider-settings';
import { useAuthStore } from '@/platform/authStore';
import type { IndexInfo, TableSchema } from '@/shared/lib/types';
import { PROVIDER_SETTINGS, connectionNeedsSecret } from '@/shared/lib/provider-settings';
import { PROVIDER_SETTINGS, needsSessionPassword } from '@/shared/lib/provider-settings';
@huyplb
huyplb merged commit c248866 into main Oct 25, 2026
13 checks passed
@huyplb
huyplb deleted the feat/redis-tables branch October 25, 2026 17:55
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant