Repository navigation
Pr/74 - #91
Merged
Merged
Pr/74#91
Conversation
The web model occasionally degenerates and emits DSML tool-call markers with a stray character between the fullwidth bars, e.g. "<｜触｜DSML｜｜ calls>", or floods bare "</｜｜DSML｜｜>" closers. These variants don't match the structured DSML regexes (which expect clean ｜｜DSML), so they leak into the reply text (seen live via a chat client). Add a catch-all `_FULLWIDTH_PIPE_TAG_RE` that strips any `<...>` token containing a fullwidth pipe U+FF5C — which never appears in legitimate prose — in the prose/leftover paths (`_strip_stray_markers`, `_strip_wrapper_tags`), i.e. after tool extraction. ASCII '|' is deliberately left untouched (tables, code). The streaming holdback in `StreamToolParser.feed` also keeps an unclosed '<' tail that already contains a U+FF5C, so a split corrupted marker is never emitted as prose before the catch-all can strip it. Tests cover the prose, streaming, split-across-chunks, and ASCII-pipe-preserved cases. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The catch-all cleaned leaked DSML markers from reply prose, but corrupted markers also land INSIDE tool-call argument values — e.g. a path parsed as "shell/core/Theme.qml</｜｜DSML｜｜ >" — which makes the tool call fail (file not found). Sanitize every string argument in `normalize_tool_call` (all tool calls route through it) with the same fullwidth-pipe catch-all. Only strings that actually contain U+FF5C are touched (recurses into dicts/lists), so normal arguments and ASCII '|' are untouched. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…e text The model occasionally emitted a raw shell command or script as its normal message content (shown to the user but never executed) instead of a tool call — e.g. a bare "cd … && python3 - <<'EOF' … EOF" delivered straight to the chat. The DSML format block alone did not rule this out. Add an explicit directive: to run a shell command, Python or any code the model MUST use a tool call, and must never write a command/script/code block as message text. Prompt-level, so it also helps the streaming path (changes what the model emits, not parsing). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Group the `functions` imports consistently in `app.py` and place the test helper import before `functions` in the incomplete-status tests.
Owner
|
@merfiDEV kindly resolve merge conflict |
Owner
|
And kindly ensure while fixing conflicts you don't break any ruff check for cleaner codebase |
Owner
|
@merfiDEV kindly fix conflicts |
- Resolved conflicts in app.py (added reply_id import) - Resolved conflicts in tests/test_dsml_leaks.py (kept both test sets) - Includes upstream hardening fixes (PR AmanCode22#94) - All tests passing (26 tests in test_dsml_leaks.py) - All ruff checks passing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
its a pr #74 but ruff fixed