Skip to content

Sync engine only supports POST + JSON body — no dynamic URL query parameters #67

Description

@eumaninho54

Status: Not started
Priority: P2 (does not block the MVP, but limits which APIs the sync engine can integrate with)
Area: C++ (core) — cpp/http/SyncHttpCaller.cpp, cpp/http/HttpUrlBuilder.cpp, src/types/sync/IEndpointDefinition.ts, src/types/sync/IRequestDefinition.ts

Description

The sync contract (ISyncDefinition) can only build the request body today via request.body (Record<string, RequestExpression>, evaluated by RequestExpressionEvaluator), and that body is always sent as a stringified JSON payload, regardless of HTTP method:

// cpp/http/SyncHttpCaller.cpp
HttpRequest request{
  methodFromString(endpoint.getString("method")),
  HttpUrlBuilder::build(network.baseUrl, path),  // path is a static string from the schema
  std::move(headers),
  json::stringify(body),                          // body is always JSON, even on GET
  network.timeoutMs
};

And endpoint.path (IEndpointDefinition.path: string) is a fixed string — no $ref/interpolation, and there is no field at all to declare query string parameters.

This means the engine can only integrate with APIs that follow a "POST with a JSON body containing cursor/operations/pageSize" shape. It cannot integrate with any API that expects filter/pagination parameters in the URL via GET, which is an extremely common pattern for "fetch what changed since X" endpoints:

  • GET /events?created[gt]=<timestamp> (Stripe-like pattern)
  • GET /products?updated_at_min=<timestamp> (Shopify-like pattern)
  • GET /items?cursor=<cursor>&limit=<pageSize> (cursor-based pagination via query instead of body)
  • Any OData-like filter ($filter=Field gt Value) or similar query string convention

In all of these cases, there is currently no way for the schema to declare "this goes in the URL," nor a way to avoid sending a JSON body on a GET (which many servers/proxies either ignore or reject outright).

Why this matters

The MVP was designed assuming POST + JSON body as the only request shape (see docs/project.md's fixed-decisions table: Transport: REST only). But "REST" covers far more request shapes than that — in particular, the "read via GET with a query string filter" pattern is at least as common as POST+body for the incremental-pull use case, arguably more so. Without this support, the sync engine is restricted to backends specifically designed for it, and cannot be wired up to existing REST APIs that follow this shape.

Possible approaches (not decided)

  • Add endpoint.query: Record<string, RequestExpression> (analogous to the existing request.body), evaluated the same way and serialized as a URL-encoded query string by HttpUrlBuilder.
  • RequestExpression today only has $ref (variable), value (constant), object, and items — none of them support concatenating a string with a variable in the middle (e.g. "LastUpdateDate gt {{cursor}}"). This likely needs a new expression variant, something like { template: string } with {{variable}} interpolation, since the final value of a query parameter is usually a composed string, not a JSON object.
  • SyncHttpCaller::send should skip (or send empty) body when endpoint.method === "GET", since sending a JSON body on GET doesn't make sense for most servers.
  • HttpUrlBuilder::build would need a new responsibility: building the query string from an evaluated key/value map, including correct URL-encoding of each value.

Suggested acceptance criteria (not finalized)

  • Design decision recorded for the shape of the new string-interpolation expression type (template, or another name).
  • IEndpointDefinition/IRequestDefinition (TS) and the corresponding parsing/evaluation (C++) updated to support endpoint.query.
  • HttpUrlBuilder correctly builds the final URL with a query string (existing + new params, URL-encoded).
  • SyncHttpCaller does not send a body (or sends an empty one) on GET requests.
  • Native test covering a full triggerSync cycle against a GET endpoint with dynamic query parameters (cursor/timestamp), through the real JSI bridge — no mocks, per cpp/tests/sync/README.md.

Related

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions