fix: extract text from tiptap bodies, and read the right description field#4
Merged
Conversation
…field
Checking the library against the live API disproved what its comments
claimed. Comment and Description were handing back raw JSON blobs, and
the parser this package has always carried could not have prevented it.
DeviantArt no longer stores rich text as Draft.js. It uses tiptap:
Draft.js, what we parse: {"blocks":[{"text":"..."}]}
tiptap, what DA sends: {"version":1,"document":{"type":"doc",...}}
Across 34 live comments: 33 tiptap, 0 Draft.js, and 33 whose .Comment was
the raw JSON. flattenMarkup now walks the tiptap tree — text nodes joined
per block, hardBreak as a newline, entities decoded — and keeps the
Draft.js path for old bodies that still carry it. Nodes with no text of
their own (images, galleries, emotes) drop out, as they always did.
The description had a second, larger bug behind that one. It lives in
Extended.DescriptionText, not TextContent, which GetDeviation read: of 24
deviations sampled, 23 populated the former and 1 the latter. So
Post.Description was empty for nearly every deviation, and the txt[1]
guard removed in 3dad0ea was never what stood in the way. Prefer
Extended.DescriptionText and fall back, since either may be the one set.
Verified against the live API rather than by reading: comments still raw
JSON went 33/34 to 0/34, and descriptions that were "" now return prose.
Also corrects the doc comments that asserted the Draft.js story on
pkg.go.dev, and notes that flattening drops images, links, and emotes.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Checking the library against the live API disproved what its own comments claimed
— including the ones I wrote in #2.
CommentandDescriptionhand back rawJSON blobs, and the parser this package has always carried could not have
prevented it.
DeviantArt does not use Draft.js anymore
It uses tiptap:
Across 34 live comments on one deviation: 33 tiptap, 0 Draft.js, and 33
whose
.Commentwas the raw JSON. The old parser matched the{...}shape,looked for
"blocks", found none, and returned its input unchanged.flattenMarkupnow walks the tiptap tree — text nodes joined per block,hardBreakas a newline, HTML entities decoded — and keeps the Draft.js path forolder bodies that still carry it. Nodes with no text of their own (images,
galleries, emotes) drop out, as they always did.
The description had a second, larger bug behind that one
It lives in
Extended.DescriptionText, not theTextContentthatGetDeviationread. Of 24 deviations sampled, 23 populated the former and 1 the latter — so
Post.Descriptionwas empty for nearly every deviation, regardless of parsing.The
txt[1] == '{'guard removed in 3dad0ea was never what stood in the way; Imisdiagnosed that, and the "behavior change" I warned about in #2 never happened.
Now prefers
Extended.DescriptionTextand falls back, since either may be set.Verified against the live API, not by reading
.Commentis raw JSON""Entities decode (
I’d→I’d), and the"version":"1"string case — DAsends
versionas both a number and a string — parses. That one matters: anytyped field for
versionfails to unmarshal on a fifth of bodies, so it isdeliberately not modelled.
Docs
Corrects the comments that asserted the Draft.js story on pkg.go.dev, and records
that flattening drops images, links, and emotes — read
TextContentfor those.18 tests pass;
go vetandgolangci-lintclean.