Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 3 additions & 5 deletions docs/02_sync_protocol.md
Original file line number Diff line number Diff line change
Expand Up @@ -407,7 +407,6 @@ first-class conflict primitives.

## Ignore lists


The `ignore` option hides path segments from the pull. Excluded
paths are still written and read inside the container — the bytes just
never cross the wire back to the DO. This is essential for any large
Expand All @@ -416,10 +415,9 @@ directory of derived files: `node_modules`, `.next`, `target`,
push tens of thousands of small files through the sync wire on the
next pull.

The default is `["node_modules"]`, applied server-side when `ignore` is
omitted. A caller-supplied list **replaces** the default — it does not
extend it. Pass `[]` to disable ignoring entirely, or pass your full
list (including `"node_modules"` if you still want it) to customise.
No paths are ignored by default. A sync server can configure an ignore list,
and an individual `fetchChanges` call can replace it. Pass `[]` to override a
server-level list and request every path.

### Ignored entries

Expand Down
2 changes: 1 addition & 1 deletion packages/dofs/src/sync/ignore.ts
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@
// node_modules_old or my_node_modules. Patterns are plain strings,
// not globs; we can extend to globs later if a real case demands it.

export const DEFAULT_IGNORE = ["node_modules"];
export const DEFAULT_IGNORE: string[] = [];

export function isIgnored(path: string, patterns: string[]): boolean {
if (patterns.length === 0) return false;
Expand Down
23 changes: 20 additions & 3 deletions packages/rpc/src/sync-driver.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -88,15 +88,31 @@ function trackFetchDisposal(
}

describe("sync driver — pullOnce", () => {
it("pulls a single entry from upstream", async () => {
it("pulls every path by default, including node_modules", async () => {
const a = makePeer();
const b = makePeer();
try {
const provider = new SQLiteWorkspaceProvider(a.db, { now: () => 1 });
provider.writeFileSync("/hello.txt", "hello");
provider.mkdirSync("/node_modules/pkg", { recursive: true });
provider.writeFileSync("/node_modules/pkg/index.js", "dependency");

const applied = await pullOnce(b.db, a.rpc);
expect(applied.applied).toBe(1);
let requestedIgnore: string[] | undefined;
const remote = new Proxy(a.rpc as object, {
get(target, prop, receiver) {
if (prop === "fetchChanges") {
return (input: Parameters<SyncRPC["fetchChanges"]>[0]) => {
requestedIgnore = input.ignore;
return Reflect.get(target, prop, receiver).call(target, input);
};
}
return Reflect.get(target, prop, receiver);
},
}) as SyncRPC;

const applied = await pullOnce(b.db, remote);
expect(requestedIgnore).toEqual([]);
expect(applied.applied).toBeGreaterThan(1);
expect(applied.skipped).toEqual([]);
expect(fileEntries(b.db)).toContain("hello.txt");
// Asserting the bytes arrived, not just the dirent. The
Expand All @@ -105,6 +121,7 @@ describe("sync driver — pullOnce", () => {
// empty on the receiver — RPC reads landed HTTP 200 / 0 bytes.
const providerB = new SQLiteWorkspaceProvider(b.db, { now: () => 1 });
expect(providerB.readFileSync("/hello.txt", "utf8")).toBe("hello");
expect(providerB.readFileSync("/node_modules/pkg/index.js", "utf8")).toBe("dependency");
} finally {
a.close();
b.close();
Expand Down
4 changes: 3 additions & 1 deletion packages/rpc/src/sync-driver.ts
Original file line number Diff line number Diff line change
Expand Up @@ -103,7 +103,9 @@ async function pullOnceImpl(
// invariant trip, and any throw inside the batch loop. Disposing the
// envelope tears down the contained stream stub, releasing the
// remote iterator.
const fetchResult = await remote.fetchChanges({ after });
// Pull the complete remote tree. Sending [] explicitly also overrides the
// historical wsd default that filtered node_modules when ignore was omitted.
const fetchResult = await remote.fetchChanges({ after, ignore: [] });
try {
const { currentCursor, appliedPushCursor } = fetchResult;
// Cross-side watermark divergence. Two shapes are recoverable:
Expand Down
Loading