Skip to content

up db apply: 500 Internal Server Error on every existing/pre-deployed item — new deploys succeed #52

Description

@rodneyjoyce

Summary

rayfin up db apply (and the db apply step inside rayfin up) reliably fails with a 500 Internal Server Error when applying any non-destructive schema config to an existing, previously-deployed Rayfin item. A brand-new item, deployed in the same session with the identical schema, succeeds every time — including a same-session re-apply (upgrade) of that new item. The failure is 100% reproducible across two completely independent, unrelated Fabric items in our tenant, and is unaffected by --force, capacity size, or which/how many entities are in the config.

We dug into the CLI source and replicated the raw HTTP call outside the CLI (bypassing its error-message truncation) to get the server's full response body, which shows the failure is actually an internal Power BI API error surfacing through the BaaS workload backend.

Environment

  • @microsoft/rayfin-cli: 1.33.2
  • @microsoft/rayfin-core: 1.33.2
  • @microsoft/rayfin-tools-common: 1.33.2
  • Node: v24.17.0 (Windows)
  • Fabric region: Australia East
  • Fabric tenant: c986e767-07af-4c9c-b2a1-08446a3c3e71

Reproduction

  1. Have an existing Rayfin item, deployed some time ago and redeployed/updated at least once since (i.e. not a same-session fresh deploy).
  2. Make any schema change (or even none — a byte-identical re-apply reproduces it too) to the @entity() classes under rayfin/data/.
  3. Run rayfin up db apply (with or without --force) against that item.
  4. Observe: 500 Internal Server Error.

Contrast:

  1. rayfin up --workspace-id <brand-new-workspace> (a workspace that has never had a Rayfin item) — succeeds, including the DB apply step.
  2. Immediately re-running rayfin up db apply against that same brand-new item (schema version 1 → 2, an in-session "upgrade") — also succeeds.

So the distinguishing factor is not new-vs-upgrade in the abstract — a same-session upgrade of a fresh item is fine. It's specifically items with real deployment history that fail.

Full raw response (captured via a manual az rest call replicating the CLI's exact request, to get the untruncated body)

The CLI's own error handling (utils/http-client.js -> throwIfNotOk()) only surfaces json.message, discarding the rest of the response. The full body is much more informative:

{
  "code": "InternalServerError",
  "subCode": 0,
  "message": "An internal error occurred.",
  "timeStamp": "2026-07-19T22:59:28.7107254Z",
  "httpStatusCode": 500,
  "hresult": -2147467259,
  "details": [
    { "code": "RootActivityId", "message": "7aad87ed-f385-47a7-ad32-616bc4183270" },
    { "code": "PowerBIApiErrorResponse", "message": "{\"error\":{\"code\":\"InternalServerError\",\"message\":\"An internal execution error occured\",\"pbi.error\":{\"code\":\"InternalServerError\",\"details\":[{\"code\":\"InternalServerError\",\"detail\":{\"type\":1,\"value\":\"An internal execution error occured\"}}]}}}" },
    { "code": "Param1", "message": "An internal execution error occured" }
  ]
}

Identical PowerBIApiErrorResponse signature reproduced on a second, completely unrelated item in the same tenant (different workspace, different item, different schema/data):

{
  "code": "InternalServerError", "subCode": 0, "message": "An internal error occurred.",
  "timeStamp": "2026-07-19T23:00:05.3102531Z", "httpStatusCode": 500, "hresult": -2147467259,
  "details": [
    { "code": "RootActivityId", "message": "755718ad-a2d9-4fb8-9df4-f1fdb9b8c40d" },
    { "code": "PowerBIApiErrorResponse", "message": "{\"error\":{\"code\":\"InternalServerError\",\"message\":\"An internal execution error occured\",\"pbi.error\":{\"code\":\"InternalServerError\",\"details\":[{\"code\":\"InternalServerError\",\"detail\":{\"type\":1,\"value\":\"An internal execution error occured\"}}]}}}" },
    { "code": "Param1", "message": "An internal execution error occured" }
  ]
}

Response headers of note: x-ms-routing-hint: host005_baas-003, x-ms-workload-resource-moniker: <echoed item id>.

Request replicated exactly as the CLI builds it:

POST https://api.fabric.microsoft.com/v1/workspaces/{workspaceId}/appBackends/{itemId}/__private/applyconfig
X-Ms-Workload-Resource-Moniker: {itemId}
Authorization: Bearer <Fabric token>
Content-Type: application/json

{"ConfigData": "<dab-config.json contents>", "Force": <bool>}

This tells us the BaaS backend, while applying config to a pre-existing item, makes an internal call into the Power BI REST API (most likely to reconcile/refresh the item's already-provisioned SQL Database / dataset binding) — and that internal Power BI-side call itself throws a generic, opaque InternalServerError with no further detail.

Bisection: it's not schema-content-specific — it's the "real commit" path itself

To rule out a specific entity or field being the trigger, we disabled all 21 @entity() classes except one (Todo), then re-enabled them one at a time, running up db apply without --force after each addition (so nothing could actually be written/dropped):

Step Entities active Result
1 Todo only 400 Bad Request — "Performing Drop table 'X' migration would result in data loss but force mode is not enabled" (expected, safe destructive-change guard)
2-20 incrementally added Same pattern every time — 400, naming whichever table is still missing
21 (full set restored) all 21 entities 500 Internal Server Error

Every partial config was safely blocked by the destructive-change guard before reaching real execution. Only the full, non-destructive config (zero drops needed) got past that guard into the real commit path — and that's exactly when it 500s, every time, regardless of which entities are involved. This indicates the bug is in the commit/apply path itself (specifically whatever step calls into the Power BI API), not in schema validation or a specific entity definition.

Impact

Any customer/dev item that has gone through more than one real deploy cycle can no longer have its database schema updated via rayfin up db apply (or rayfin up) at all — every attempt fails with an opaque 500, both with and without --force. This blocks all further schema iteration on any Rayfin app once it's past its first deploy. We worked around nothing — this is currently a hard blocker for us.

What we've ruled out

  • Capacity/throttling — reproduced identically after scaling the target capacity up to F16.
  • Schema/entity content — reproduced with a single trivial entity's worth of "real" config (once it passes the destructive-change guard) just as with the full 21-entity config; also reproduced on two schemas that differ in real customer data/usage history.
  • Item-specific corruption — reproduced on two unrelated items in different workspaces.
  • The CLI itself mis-forming the request — replicated the exact same request manually via az rest, same result.
  • Destructive-change blocking mis-surfacing as 500 — the destructive-change guard correctly returns 400 with a clear message when triggered; the 500 only happens on the path that has already passed that guard.

Ask

Could someone with visibility into the BaaS/DAB backend's Power BI API integration look at the RootActivityIds above (or ask us for more — we can reproduce this on demand)? Given the pattern (any real execution against an item with actual deployment history fails; a same-session-fresh item never does), this looks like the internal Power BI reconciliation call choking on some state that only exists after a real prior deploy — worth checking what call is being made and what's different about its target after a redeploy vs. an initial deploy.

Happy to provide the full generated dab-config.json or any other repro artifacts on request.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions