Skip to content

fix: extract text from tiptap bodies, and read the right description field#4

Merged
ccleberg merged 1 commit into
mainfrom
fix/tiptap-text-extraction
Jul 15, 2026
Merged

fix: extract text from tiptap bodies, and read the right description field#4
ccleberg merged 1 commit into
mainfrom
fix/tiptap-text-extraction

Conversation

@ccleberg

Copy link
Copy Markdown
Member

Checking the library against the live API disproved what its own comments claimed
— including the ones I wrote in #2. Comment and Description hand back raw
JSON blobs, and the parser this package has always carried could not have
prevented it.

DeviantArt does not use Draft.js anymore

It uses tiptap:

Draft.js, what we parse:  {"blocks":[{"text":"..."}]}
tiptap, what DA sends:    {"version":1,"document":{"type":"doc","content":[...]}}

Across 34 live comments on one deviation: 33 tiptap, 0 Draft.js, and 33
whose .Comment was the raw JSON
. The old parser matched the {...} shape,
looked for "blocks", found none, and returned its input unchanged.

flattenMarkup now walks the tiptap tree — text nodes joined per block,
hardBreak as a newline, HTML entities decoded — and keeps the Draft.js path for
older bodies that still carry it. Nodes with no text of their own (images,
galleries, emotes) drop out, as they always did.

The description had a second, larger bug behind that one

It lives in Extended.DescriptionText, not the TextContent that GetDeviation
read. Of 24 deviations sampled, 23 populated the former and 1 the latter — so
Post.Description was empty for nearly every deviation, regardless of parsing.
The txt[1] == '{' guard removed in 3dad0ea was never what stood in the way; I
misdiagnosed that, and the "behavior change" I warned about in #2 never happened.
Now prefers Extended.DescriptionText and falls back, since either may be set.

Verified against the live API, not by reading

before after
comments whose .Comment is raw JSON 33/34 0/34
descriptions returning "" 23/24 0/24

Entities decode (I’dI’d), and the "version":"1" string case — DA
sends version as both a number and a string — parses. That one matters: any
typed field for version fails to unmarshal on a fifth of bodies, so it is
deliberately not modelled.

Docs

Corrects the comments that asserted the Draft.js story on pkg.go.dev, and records
that flattening drops images, links, and emotes — read TextContent for those.

18 tests pass; go vet and golangci-lint clean.

…field

Checking the library against the live API disproved what its comments
claimed. Comment and Description were handing back raw JSON blobs, and
the parser this package has always carried could not have prevented it.

DeviantArt no longer stores rich text as Draft.js. It uses tiptap:

  Draft.js, what we parse:  {"blocks":[{"text":"..."}]}
  tiptap, what DA sends:    {"version":1,"document":{"type":"doc",...}}

Across 34 live comments: 33 tiptap, 0 Draft.js, and 33 whose .Comment was
the raw JSON. flattenMarkup now walks the tiptap tree — text nodes joined
per block, hardBreak as a newline, entities decoded — and keeps the
Draft.js path for old bodies that still carry it. Nodes with no text of
their own (images, galleries, emotes) drop out, as they always did.

The description had a second, larger bug behind that one. It lives in
Extended.DescriptionText, not TextContent, which GetDeviation read: of 24
deviations sampled, 23 populated the former and 1 the latter. So
Post.Description was empty for nearly every deviation, and the txt[1]
guard removed in 3dad0ea was never what stood in the way. Prefer
Extended.DescriptionText and fall back, since either may be the one set.

Verified against the live API rather than by reading: comments still raw
JSON went 33/34 to 0/34, and descriptions that were "" now return prose.

Also corrects the doc comments that asserted the Draft.js story on
pkg.go.dev, and notes that flattening drops images, links, and emotes.
@ccleberg
ccleberg merged commit 9c6ad66 into main Jul 15, 2026
9 checks passed
@ccleberg
ccleberg deleted the fix/tiptap-text-extraction branch July 15, 2026 23:02
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant