Repository navigation
test(pageshim): replay a full-size ChatGPT run and bound its bridge traffic - #275
Merged
Merged
Conversation
tnunamak
force-pushed
the
waspflow/m19-chatgpt-replay
branch
from
October 6, 2026 03:35
a8b0153 to
d2c0667
Compare
tnunamak
changed the title
tnunamak
changed the base branch from
waspflow/m8-pageshim-streaming-main
to
main
October 6, 2026 04:06
tnunamak
force-pushed
the
waspflow/m19-chatgpt-replay
branch
from
October 6, 2026 04:06
d2c0667 to
b2b4048
Compare
… replay Main's PageShim tests drive the ChatGPT bundle with at most 51 one-message conversations and no /conversations/batch endpoint, so the bridge-call savings from #340 had no end-to-end guard. A change that sent one bridge call per record or per conversation again would pass every existing test. The new test projects 600 synthetic conversations of 6 messages into provider-shaped search pages and batch details, then runs the real ChatGPT PageShim bundle in the headless harness. It checks that the run completes with the right counts, that detail reads use the batch endpoint, and that bridge traffic stays bounded: under 100 result-stream messages, under 150 setData calls, under 120 setProgress calls. Today the run makes 20 setData and 55 setProgress calls. With the pre-#340 runtime, lane and connector swapped in, it made 8,409 and 630, and the test failed. It takes about 5 s. The reverse projection module comes from the earlier replay branch. Its hardcoded path to a private retained export is gone, and a retained export now needs an explicit path. The bridge-latency model, the AsyncLocalStorage shim, REPORT.md and the replay runner are dropped: main has no AsyncLocalStorage dependency, and the measurements were one-off. pageshim.yml now runs the new test file. Assisted-by: AI Signed-off-by: Tim Nunamaker <tnunamak@gmail.com>
tnunamak
force-pushed
the
waspflow/m19-chatgpt-replay
branch
from
October 6, 2026 04:21
b2b4048 to
a20f81d
Compare
tnunamak
marked this pull request as ready for review
October 6, 2026 04:37
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The ChatGPT PageShim tests on main cannot catch a bridge-call regression. They run at most 51 one-message conversations, and their fixture has no
/conversations/batchendpoint. The chunk-packing test from #340 uses a synthetic entry, not the ChatGPT bundle. So a change that sends one bridge call per record or per conversation again would pass every existing test. On a real 2,042-conversation account that cost about 135,000setDatacalls and made the import 2.6× slower than the legacy connector.This adds a full-size replay of the real ChatGPT PageShim bundle in the headless harness, built from synthetic data. It projects 600 conversations of 6 messages into provider-shaped search pages and batch details, runs the bundle, and checks that the run completes with the exact counts, reads details through the batch endpoint, and keeps bridge traffic bounded: under 100 result-stream messages, under 150
setDataand under 120setProgresscalls. Main makes 20setData, 18 stream messages and 55setProgresscalls. With the pre-#340 runtime, lane and connector swapped in, the run made 8,409, 8,407 and 630 and the test failed. The test takes about 5 s.The reverse-projection module comes from the earlier replay branch, with the hardcoded path to a private retained export removed. A retained export now needs an explicit path. The bridge-latency model, the AsyncLocalStorage shim,
REPORT.mdand the replay runner are dropped: main has no AsyncLocalStorage dependency, and the measurements were one-off.pageshim.ymlnow runs the new test file, since that step lists test files by name.This PR does not change
pageshim-streaming.test.mjs, which that CI step does not run (found while checking). It does not change any connector, so there is no version bump.Assisted-by: AI