Repository navigation
fix: key collisions and logprobs lost on replayed streams - #11
Merged
Merged
Conversation
The canonical copy was a plain object, so an own __proto__ key replaced the prototype and vanished, and it ignored toJSON, so every Date hashed as {}. Keys now use a null-prototype copy, follow toJSON like JSON.stringify, and treat other non-plain objects (Map, Set, class instances) as unkeyable. deriveKey turns any failure, including BigInt and cycles, into a bypass so the call still reaches the model. Plain JSON hashes are unchanged (pinned by a test), so KEY_VERSION stays 1.
The stream assembler stored logprobs as null and kept only id, created and model from the chunks, so a cached stream answered a logprobs request without them. It now accumulates logprobs per choice like ChatCompletionStream (first copied, later content appended) and keeps system_fingerprint and service_tier, and the replay emits them.
From code review. The catch now covers only serialization, so a malformed input still throws instead of turning into a silent bypass, and the reason for a bypass goes to onError with a new key stage. Plain objects from another realm (vm, jsdom, some edge runtimes) are hashed again, toJSON is applied once per value as JSON.stringify does, and the comments say why class instances are rejected.
The replay checked service_tier by truthiness, so a null the stream carried came back absent. It now checks for undefined, and a comment records why refusal logprobs are never appended.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Two defects found by an external review of
c726aa4. I reproduced both before fixing them, and the plan review found a third.{}: an own__proto__key replaced the prototype and vanished, andtoJSONwas ignored, so everyDatehashed as{}.toJSONfollowed likeJSON.stringify(only when it is a function); other non-plain objects (Map,Set,RegExp, class instances) are unkeyable and bypass the cache.BigIntor a cycle inparamsmadewrap()reject instead of calling the modelderiveKeythrew and no caller caught it, breaking fail-open.deriveKeyturns any failure into a bypass.logprobs,system_fingerprintandservice_tierlogprobs: nulland kept onlyid/created/model.ChatCompletionStream(firstlogprobscopied, later content appended; last value wins for the top-level fields) and replay them.KEY_VERSIONstays 1. Plain JSON, which is what the SDK wrappers send, hashes exactly as before. A test pins the hash computed onc726aa4for a typical request with system prompt, prior turns, tools and nested params. Only inputs that used to collide get new hashes; their old rows simply miss.Testing
{}copy fails the__proto__test;logprobsfails the replay test.ChatCompletionStream.finalChatCompletion().Code review follow-up
An independent code review found six more points, fixed in 8831524 and 65e89c8:
onErrorwith a newkeystage;vm, jsdom) are hashed again;toJSONis applied once per value, asJSON.stringifydoes;nullservice_tiersurvives replay;Each fix has a test, and the review re-checked both commits. The golden hash still holds. CI green on 65e89c8.