Repository navigation
fix(spotify): empty rankings export as collected; offset-0 recently-played 404 is unavailable, not empty (0.1.57) - #635
Merged
Conversation
…y-played 404 is unavailable, not empty (0.1.57)
Since 0.1.56, runs that asked only for recently_played_contexts,
top_artists and top_tracks failed with connector_reported_failure
"Spotify did not provide a request-bound terminal boundary; no checkpoint
was saved." (8 runs, 3 users, 16:42Z to 17:46Z on 8 Nov). Those users had
just run the full set, seen those three scopes omitted, and retried them.
That message was not the cause. The connector always ends
recently_played_contexts with a boundary skip and each ranking with a
ranked_list_capped skip. With zero rows those skips make the scope omitted,
and when every requested scope is omitted PageShim reports the first skip's
text as the run error. The rows were zero because:
- userTopContent answered totalCount 0 in both windows. That is a complete,
empty ranking, not a capped one, but nothing said so.
- recently-played answered 404 with an empty body at offset 0, which 0.1.55
reads as an empty history. It is not reliable: one user got it on four
runs in a row, then a page of contexts on the next run a minute later.
Changes:
- A ranking whose total is 0 in every window saves a full-scan STATE and
no capped skip, so it exports as an empty, produced scope (as playlists
do since 0.1.56). Diagnostic ranked_list_empty {stream}.
- An offset-0 recently-played 404 ends the stream with the new retryable
skip spotify_recently_played_unavailable instead of the boundary skip.
Diagnostic recently_played_unavailable {endedBy, othersRequested}.
- A run that asked for recent + rankings now returns partial (rankings
produced empty, recent omitted) instead of failing. A run that asked only
for recently played and got the 404 still fails, with the honest,
retryable reason: nothing was collected.
- Non-empty rankings and recent history are unchanged.
- Regenerated the library metadata and the implementation index (which
also picks up 0.1.56's missing reason messages); allowlist lines moved.
Live run in a logged-in open.spotify.com tab, RSS classifier, with the
404 and the empty rankings simulated in the fetch shim:
- 3 scopes: main failure (produced 0, omitted 3); fix partial (produced 2,
omitted 1).
- top lists only: main failure; fix success (produced 2, empty).
- recent only: failure on both; the fix's reason is the new one.
- Real, unsimulated 7 scopes and 3 scopes: identical outcome and row
counts on main and the fix.
Claude-Session: https://claude.ai/code/session_013cAazvHSyzKgPDbCjuZ4gA
Signed-off-by: kahtaf <kahtaf@gmail.com>
github-actions
Bot
added
the
evidence: none
…empty rankings and an unavailable recent history Claude-Session: https://claude.ai/code/session_013cAazvHSyzKgPDbCjuZ4gA Signed-off-by: kahtaf <kahtaf@gmail.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What broke
Since 0.1.56 hit RSS prod (~15:55Z, 8 Nov), 8 runs from 3 users (16:42Z to 17:46Z) failed with
connector_reported_failure: "Spotify did not provide a request-bound terminal boundary; no checkpoint was saved." Every one of them asked only forrecently_played_contexts+top_artists+top_tracks(orrecently_played_contextsalone). These were retries. Each user had just run the full set, got those three scopes back omitted (produced 4 of 7), and asked for them again.Root cause
The message is a red herring. The connector always ends
recently_played_contextswith thespotify_recent_context_boundary_unverifiedskip, and each ranking withspotify_ranked_list_capped. When a stream has zero rows, that skip makes it omitted. When every requested scope is omitted, PageShim throws the first skip's text as the run error (runtime.ts:errors[0]?.reason).The rows were zero for two reasons, both visible in the prod
[spotify-diagnostic]lines and run errors (no record content was read):userTopContentreturnedtotalCount: 0in both windows. Nostream_failedor withheld rows. That is a complete, empty ranking, but the connector still called it "capped" and saved no checkpoint, so it could never export as "collected, none".recently_played_end {"endedBy":"not_found_at_start"}). Since 0.1.55 this has been read as an empty history. It isn't reliable. User267493…got it on 4 runs in a row (17:42Z to 17:46Z), then a page of contexts at 17:47Z. Across all runs since the deploy, 15 of 20 recently-played reads ended this way.Fix (0.1.57)
STATEand no capped skip, so it exports as an empty, produced scope (the same way playlists do since 0.1.56). New diagnostic:ranked_list_empty {stream}.spotify_recently_played_unavailable(retry_by_runtime), instead of the boundary skip. It is reported as unavailable, not as proven empty. New diagnostic:recently_played_unavailable {endedBy, othersRequested}. It carries no user data.reason_display_messages. Ranpolyfill-library-metadata:generateandconnector-implementation-index:generate; the index also picks up 0.1.56's missing reason messages. Line numbers in the no-await-in-loops allowlist moved.A run that asks only for recently played and gets the 404 still fails, now with the honest, retryable reason. Nothing was collected, so there is nothing to return.
Live test
Built bundles of
main(0.1.56) and this branch, ran them in a logged-in open.spotify.com tab, and classified the results with RSS'sresult-classifier.cjs. The 404 and the empty rankings were simulated in the fetch shim.Existing Spotify tests pass (150/150 connector, 21/21 Spotify PageShim). No new unit tests, per request. One existing PageShim assertion (the empty-account case) pinned the old behaviour, empty rankings omitted and 4 streamed scopes. It now expects 6 scopes, with the rankings produced empty and recent history unavailable.
https://claude.ai/code/session_013cAazvHSyzKgPDbCjuZ4gA