Status: Not started
Priority: P2 (does not block the MVP, but limits which APIs the sync engine can integrate with)
Area: C++ (core) — cpp/http/SyncHttpCaller.cpp, cpp/http/HttpUrlBuilder.cpp, src/types/sync/IEndpointDefinition.ts, src/types/sync/IRequestDefinition.ts
Description
The sync contract (ISyncDefinition) can only build the request body today via request.body (Record<string, RequestExpression>, evaluated by RequestExpressionEvaluator), and that body is always sent as a stringified JSON payload, regardless of HTTP method:
// cpp/http/SyncHttpCaller.cpp
HttpRequest request{
methodFromString(endpoint.getString("method")),
HttpUrlBuilder::build(network.baseUrl, path), // path is a static string from the schema
std::move(headers),
json::stringify(body), // body is always JSON, even on GET
network.timeoutMs
};
And endpoint.path (IEndpointDefinition.path: string) is a fixed string — no $ref/interpolation, and there is no field at all to declare query string parameters.
This means the engine can only integrate with APIs that follow a "POST with a JSON body containing cursor/operations/pageSize" shape. It cannot integrate with any API that expects filter/pagination parameters in the URL via GET, which is an extremely common pattern for "fetch what changed since X" endpoints:
GET /events?created[gt]=<timestamp> (Stripe-like pattern)
GET /products?updated_at_min=<timestamp> (Shopify-like pattern)
GET /items?cursor=<cursor>&limit=<pageSize> (cursor-based pagination via query instead of body)
- Any OData-like filter (
$filter=Field gt Value) or similar query string convention
In all of these cases, there is currently no way for the schema to declare "this goes in the URL," nor a way to avoid sending a JSON body on a GET (which many servers/proxies either ignore or reject outright).
Why this matters
The MVP was designed assuming POST + JSON body as the only request shape (see docs/project.md's fixed-decisions table: Transport: REST only). But "REST" covers far more request shapes than that — in particular, the "read via GET with a query string filter" pattern is at least as common as POST+body for the incremental-pull use case, arguably more so. Without this support, the sync engine is restricted to backends specifically designed for it, and cannot be wired up to existing REST APIs that follow this shape.
Possible approaches (not decided)
- Add
endpoint.query: Record<string, RequestExpression> (analogous to the existing request.body), evaluated the same way and serialized as a URL-encoded query string by HttpUrlBuilder.
RequestExpression today only has $ref (variable), value (constant), object, and items — none of them support concatenating a string with a variable in the middle (e.g. "LastUpdateDate gt {{cursor}}"). This likely needs a new expression variant, something like { template: string } with {{variable}} interpolation, since the final value of a query parameter is usually a composed string, not a JSON object.
SyncHttpCaller::send should skip (or send empty) body when endpoint.method === "GET", since sending a JSON body on GET doesn't make sense for most servers.
HttpUrlBuilder::build would need a new responsibility: building the query string from an evaluated key/value map, including correct URL-encoding of each value.
Suggested acceptance criteria (not finalized)
Related
Status: Not started
Priority: P2 (does not block the MVP, but limits which APIs the sync engine can integrate with)
Area: C++ (core) —
cpp/http/SyncHttpCaller.cpp,cpp/http/HttpUrlBuilder.cpp,src/types/sync/IEndpointDefinition.ts,src/types/sync/IRequestDefinition.tsDescription
The sync contract (
ISyncDefinition) can only build the request body today viarequest.body(Record<string, RequestExpression>, evaluated byRequestExpressionEvaluator), and that body is always sent as a stringified JSON payload, regardless of HTTP method:And
endpoint.path(IEndpointDefinition.path: string) is a fixed string — no$ref/interpolation, and there is no field at all to declare query string parameters.This means the engine can only integrate with APIs that follow a "POST with a JSON body containing cursor/operations/pageSize" shape. It cannot integrate with any API that expects filter/pagination parameters in the URL via
GET, which is an extremely common pattern for "fetch what changed since X" endpoints:GET /events?created[gt]=<timestamp>(Stripe-like pattern)GET /products?updated_at_min=<timestamp>(Shopify-like pattern)GET /items?cursor=<cursor>&limit=<pageSize>(cursor-based pagination via query instead of body)$filter=Field gt Value) or similar query string conventionIn all of these cases, there is currently no way for the schema to declare "this goes in the URL," nor a way to avoid sending a JSON
bodyon aGET(which many servers/proxies either ignore or reject outright).Why this matters
The MVP was designed assuming POST + JSON body as the only request shape (see
docs/project.md's fixed-decisions table:Transport: REST only). But "REST" covers far more request shapes than that — in particular, the "read via GET with a query string filter" pattern is at least as common as POST+body for the incremental-pull use case, arguably more so. Without this support, the sync engine is restricted to backends specifically designed for it, and cannot be wired up to existing REST APIs that follow this shape.Possible approaches (not decided)
endpoint.query: Record<string, RequestExpression>(analogous to the existingrequest.body), evaluated the same way and serialized as a URL-encoded query string byHttpUrlBuilder.RequestExpressiontoday only has$ref(variable),value(constant),object, anditems— none of them support concatenating a string with a variable in the middle (e.g."LastUpdateDate gt {{cursor}}"). This likely needs a new expression variant, something like{ template: string }with{{variable}}interpolation, since the final value of a query parameter is usually a composed string, not a JSON object.SyncHttpCaller::sendshould skip (or send empty)bodywhenendpoint.method === "GET", since sending a JSON body on GET doesn't make sense for most servers.HttpUrlBuilder::buildwould need a new responsibility: building the query string from an evaluated key/value map, including correct URL-encoding of each value.Suggested acceptance criteria (not finalized)
template, or another name).IEndpointDefinition/IRequestDefinition(TS) and the corresponding parsing/evaluation (C++) updated to supportendpoint.query.HttpUrlBuildercorrectly builds the final URL with a query string (existing + new params, URL-encoded).SyncHttpCallerdoes not send abody(or sends an empty one) onGETrequests.triggerSynccycle against aGETendpoint with dynamic query parameters (cursor/timestamp), through the real JSI bridge — no mocks, percpp/tests/sync/README.md.Related
SyncHttpCaller/HttpUrlBuilder.